Network Working Group M.T. Rose
Internet-Draft Invisible Worlds, Inc.
Expires: December 28, 2000 June 29, 2000
On the Design of Application Protocols
draft-mrose-bxxp-design-00
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 except that the right to
produce derivative works is not granted. (If this document becomes
part of an IETF working group activity, then it will be brought into
full compliance with Section 10 of RFC2026.)
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 28, 2000.
Copyright Notice
Copyright (C) The Internet Society (2000). All Rights Reserved.
Abstract
This memo describes the design principles for the Blocks eXtensible
eXchange Protocol (BXXP). BXXP is a generic application protocol
framework for connection-oriented, asynchronous request/response
interactions. The framework permits multiplexing of independent
request/response streams over a single transport connection,
supporting both textual and binary messages.
Rose Expires December 28, 2000 [Page 1]
Internet-Draft On the Design of Application Protocols June 2000
Table of Contents
1. A Problem 19 Years in the Making . . . . . . . . . . . . . . . 3
2. You can Solve Any Problem... . . . . . . . . . . . . . . . . . 6
3. Protocol Mechanisms . . . . . . . . . . . . . . . . . . . . . 8
3.1 Framing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Error Reporting . . . . . . . . . . . . . . . . . . . . . . . 9
3.4 Multiplexing . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.5 User Authentication . . . . . . . . . . . . . . . . . . . . . 11
3.6 Transport Security . . . . . . . . . . . . . . . . . . . . . . 12
3.7 Let's Recap . . . . . . . . . . . . . . . . . . . . . . . . . 13
4. Protocol Properties . . . . . . . . . . . . . . . . . . . . . 14
4.1 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2 Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3 Simplicity . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.4 Extensibility . . . . . . . . . . . . . . . . . . . . . . . . 15
5. The BXXP Framework . . . . . . . . . . . . . . . . . . . . . . 17
5.1 Framing and Encoding . . . . . . . . . . . . . . . . . . . . . 17
5.2 Error Reporting . . . . . . . . . . . . . . . . . . . . . . . 19
5.3 Multiplexing . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.4 User Authentication . . . . . . . . . . . . . . . . . . . . . 21
5.5 Transport Security . . . . . . . . . . . . . . . . . . . . . . 21
5.6 Things We Left Out . . . . . . . . . . . . . . . . . . . . . . 22
6. Current Status . . . . . . . . . . . . . . . . . . . . . . . . 23
7. To Be Determined... . . . . . . . . . . . . . . . . . . . . . 24
References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Author's Address . . . . . . . . . . . . . . . . . . . . . . . 27
Full Copyright Statement . . . . . . . . . . . . . . . . . . . 28
Rose Expires December 28, 2000 [Page 2]
Internet-Draft On the Design of Application Protocols June 2000
1. A Problem 19 Years in the Making
SMTP[1] is close to being the perfect application protocol: it
solves a large, important problem in a minimalist way. It's simple
enough for an entry-level implementation to fit on one or two
screens of code, and flexible enough to form the basis of very
powerful product offerings in a robust and competitive market.
Modulo a few oddities (e.g., SAML), the design is well conceived and
the resulting specification is well-written and largely
self-contained. There is very little about good application protocol
design that you can't learn by reading the SMTP specification.
Unfortunately, there's one little problem: SMTP was originally
published in 1981 and since that time, a lot of application
protocols have been designed for the Internet, but there hasn't been
a lot of reuse going on. You might expect this if the application
protocols were all radically different, but this isn't the case:
most are surprisingly similar in their functional behavior, even
though the actual details vary considerably.
In late 1998, as Carl Malamud and I were sitting down to review the
Blocks architecture[2], we realized that we needed to have a
protocol for exchanging Blocks. The conventional wisdom is that when
you need an application protocol, there are four ways to proceed:
1. find an existing exchange protocol that (more or less) does what
you want;
2. define an exchange model on top of the world-wide web
infrastructure that (more or less) does what you want;
3. define an exchange model on top of the electronic mail
infrastructure that (more or less) does what you want; or,
4. define a new protocol from scratch that does exactly what you
want.
An engineer can make reasoned arguments about the merits of each of
the these approaches. Here's the process we followed...
The most appealing option is to find an existing protocol and use
that. (In other words, we'd rather "buy" than "make".) So, we did a
survey of many existing application protocols and found that none of
them were a good match for the semantics of the protocol we needed.
For example, most application protocols are oriented toward
client/server behavior, and emphasize the client pulling data from
the server; in contrast with Blocks, a client usually pulls data
from the server, but it also may request the server to
Rose Expires December 28, 2000 [Page 3]
Internet-Draft On the Design of Application Protocols June 2000
asynchronously push (new) data to it. Clearly, we could mutate a
protocol such as FTP[3] or SMTP into what we wanted, but by the time
we did all that, the base protocol and our protocol would have more
differences than similarities. In other words, the cost of modifying
an off-the-shelf implementation becomes comparable with starting
from scratch.
Another approach is to use HTTP[4] as the exchange protocol and
define the rules for data exchange over that. For example, IPP[5]
(the Internet Printing Protocol) uses this approach. The basic idea
is that HTTP defines the rules for exchanging data and then you
define the data's syntax and semantics. Because you inherit the
entire HTTP infrastructure (e.g., HTTP's authentication mechanisms,
caching proxies, and so on), there's less for you to have to invent
(and code!). Or, conversely, you might view the HTTP infrastructure
as too helpful. As an added bonus, if you decide that your protocol
runs over port 80, you may be able to sneak your traffic past older
firewalls, at the cost of port 80 saturation.
HTTP has many strengths: it's ubiquitous, it's familiar, and there
are a lot of tools available for developing HTTP-based systems.
Another good thing about HTTP is that it uses MIME[6] for encoding
data.
Unfortunately for us, even with HTTP 1.1[7], there still wasn't a
good fit. As a consequence of the highly-desirable goal of
maintaining compatibility with the original HTTP, HTTP's framing
mechanism isn't flexible enough to support server-side asynchronous
behavior and its authentication model isn't similar to other
Internet applications.
Mapping IPP onto HTTP 1.1 illustrates the latter issue. For example,
the IPP server is supposed to signal its client when a job
completes. Since the HTTP client must originate all requests and
since the decision to close a persistent connection in HTTP is
unilateral, the best that the IPP specification can do is specify
this functionality in a non-deterministic fashion.
Further, the IPP mapping onto HTTP shows that even subtle shifts in
behavior have unintended consequences. For example, requests in IPP
are typically much larger than those seen by many HTTP server
implementations -- resulting in oddities in many HTTP servers (e.g.,
requests are sometimes silently truncated). The lesson is that
HTTP's framing mechanism is very rigid with respect to its view of
the request/response model.
Lastly, given our belief that the port field of the TCP header isn't
a constant 80, we were immune to the seductive allure of wanting to
sneak our traffic past unwary site administrators.
Rose Expires December 28, 2000 [Page 4]
Internet-Draft On the Design of Application Protocols June 2000
The third choice, layering the protocol on top of e-mail, was
attractive. Unfortunately, the nature of our application includes a
lot of interactivity with relatively small response times. So, this
left us the final alternative: defining a protocol from scratch.
To begin, we figured that our requirements, while a little more
stringent than most, could fit inside a framework suitable for a
large number of future application protocols. The trick is to avoid
the kitchen-sink approach. (Dave Clark[39] has a saying: "One of the
roles of architecture is to tell you what you can't do.")
Rose Expires December 28, 2000 [Page 5]
Internet-Draft On the Design of Application Protocols June 2000
2. You can Solve Any Problem...
...if you're willing to make the problem small enough.
Our most important step is to limit the problem to application
protocols that exhibit certain features:
o they are connection-oriented;
o they use requests and responses to exchange messages; and,
o they allow for asynchronous message exchange.
Let's look at each, in turn.
First, we're only going to consider connection-oriented application
protocols (e.g., those that work on top of TCP[8]). Another branch
in the taxonomy, connectionless, consists of those that don't want
the delay or overhead of establishing and maintaining a reliable
stream. For example, most DNS[9] traffic is characterized by a
single request and response, both of which fit within a single IP
datagram. In this case, it makes sense to implement a basic
reliability service above the transport layer in the application
protocol itself.
Second, we're only going to consider message-oriented application
protocols. A "message" -- in our lexicon -- is simply structured
data exchanged between loosely-coupled systems. Another branch in
the taxonomy, tightly-coupled systems, uses remote procedure calls
as the exchange paradigm. Unlike the
connection-oriented/connectionless dichotomy, the issue of loosely-
or tightly-coupled systems is similar to a continuous spectrum.
Fortunately, the edges are fairly sharp.
For example, NFS[10] is a tightly-coupled system using RPCs. When
running in a properly-configured LAN, a remote disk accessible via
NFS is virtually indistinguishable from a local disk. To achieve
this, tightly-coupled systems are highly concerned with issues of
latency. Hence, most (but not all) tightly-coupled systems use
connection-less RPC mechanisms; further, most tend to be implemented
as operating system functions rather than user-level programs. (In
some environments, the tightly-coupled systems are implemented as
single-purpose servers, on hardware specifically optimized for that
one function.)
Finally, we're going to consider the needs of application protocols
that exchange messages asynchronously. The classic client/server
model is that the client sends a request and the server sends a
response. If you think of requests as "questions" and responses as
Rose Expires December 28, 2000 [Page 6]
Internet-Draft On the Design of Application Protocols June 2000
"answers", then the server answers only those questions that it's
asked and it never asks any questions of its own. We'll need to
support a more general model, peer-to-peer. In this model, for a
given transaction one peer might be the "client" and the other the
"server", but for the next transaction, the two peers might switch
roles.
It turns out that the client/server model is a proper subset of the
peer-to-peer model: it's acceptable for a particular application
protocol to dictate that the peer that establishes the connection
always acts as the client (initiates requests), and that the peer
that listens for incoming connections always acts as the server
(issuing responses to requests).
There are quite a few existing application domains that don't fit
our requirements, e.g., nameservice (via the DNS), fileservice (via
NFS), multicast-enabled applications such as distributed video
conferencing, and so on. However, there are a lot of application
domains that do fit these requirements, e.g., electronic mail, file
transfer, remote shell, and the world-wide web. So, the bet we are
placing in going forward is that there will continue to be reasons
for defining protocols that fit within our framework.
Rose Expires December 28, 2000 [Page 7]
Internet-Draft On the Design of Application Protocols June 2000
3. Protocol Mechanisms
The next step is to look at the tasks that an application protocol
must perform and how it goes about performing them. Although an
exhaustive exposition might identify a dozen (or so) areas, the ones
we're interested in are:
o framing, which tells how the beginning and ending of each message
is delimited;
o encoding, which tells how a message is represented when exchanged;
o error reporting, which tells how errors are described;
o multiplexing, which tells how independent parallel exchanges are
handled;
o user authentication, which tells how the peers at each end of the
connection are identified and verified; and,
o transport security, which tells how the exchanges are protected
against third-party interception or modification.
A notable absence in this list is naming -- we'll explain why later
on.
3.1 Framing
There are three commonly used approaches to delimiting messages:
octet-stuffing, octet-counting, and connection-blasting.
An example of a protocol that uses octet-stuffing is SMTP. Commands
in SMTP are line-oriented (each command ends in a CR-LF pair). When
an SMTP peer sends a message, it first transmits the "DATA" command,
then it transmits the message, then it transmits a "." (dot)
followed by a CR-LF. If the message contains any lines that begin
with a dot, the sending SMTP peer sends two dots; similarly, when
the other SMTP peer receives a line that begins with a dot, it
discards the dot, and, if the line is empty, then it knows it's
received the entire message. Octet-stuffing has the property that
you don't need the entire message in front of you before you start
sending it. Unfortunately, it's slow because both the sender and
receiver must scan each line of the message to see if they need to
transform it.
An example of a protocol that uses octet-counting is HTTP. Commands
in HTTP consist of a request line followed by headers and a body.
The headers contain an octet count indicating how large the body is.
The properties of octet-counting are the inverse of octet-stuffing:
Rose Expires December 28, 2000 [Page 8]
Internet-Draft On the Design of Application Protocols June 2000
before you can start sending a message you need to know the length
of the whole message, but you don't need to look at the content of
the message once you start sending or receiving.
An example of a protocol that uses connection-blasting is FTP.
Commands in FTP are line-oriented, and when it's time to exchange a
message, a new TCP connection is established to transmit the
message. Both octet-counting and connection-blasting have the
property that the messages can be arbitrary binary data; however,
the drawback of the connection-blasting approach is that the peers
need to communicate IP addresses and TCP port numbers, which may be
"transparently" altered by NATS[11] and network bugs. In addition,
if the messages being exchanged are small (say less than 32k), then
the overhead of establishing a connection for each message
contributes significant latency during data exchange.
3.2 Encoding
There are many schemes used for encoding data (and many more
encoding schemes have been proposed than are actually in use).
Fortunately, only a few are burning brightly on the radar.
The messages exchanged using SMTP are encoded using the
822-style[12]. The 822-style divides a message into textual headers
and an unstructured body. Each header consists of a name and a value
and is terminated with a CR-LF pair. An additional CR-LF separates
the headers from the body.
It is this structure that HTTP uses to indicate the length of the
body for framing purposes. More formally, HTTP uses MIME, an
application of the 822-style to encode both the data itself (the
body) and information about the data (the headers). That is,
although HTTP is commonly viewed as a retrieval mechanism for
HTML[13], it is really a retrieval mechanism for objects encoded
using MIME, most of which are either HTML pages or referenced
objects such as GIFs.
3.3 Error Reporting
An application protocol needs a mechanism for conveying error
information between peers. The first formal method for doing this
was defined by SMTP's "theory of reply codes". The basic idea is
that an error is identified by a three-digit string, with each
position having a different significance:
the first digit: indicating success or failure, either permanent or
transient;
the second digit: indicating the part of the system reporting the
Rose Expires December 28, 2000 [Page 9]
Internet-Draft On the Design of Application Protocols June 2000
situation (e.g., the syntax analyzer); and,
the third digit: identifying the actual situation.
Operational experience with SMTP suggests that the range of error
conditions is larger than can be comfortably encoded using a
three-digit string (i.e., you can report on only 10 different things
going wrong for any given part of the system). So, [14] provides a
convenient mechanism for extending the number of values that can
occur in the second and third positions.
Virtually all of the application protocols we've discussed thus far
use the three-digit reply codes, although there is less coordination
between the designers of different application protocols than most
would care to admit. (A notable exception to the theory of reply
codes is IMAP[15] which uses error "tokens" instead of three-digit
codes.)
In addition to conveying a reply code, most application protocols
also send a textual diagnostic suitable for human, not machine,
consumption. (More accurately, the textual diagnostic is suitable
for people who can read a widely used variant of the English
language.) Since reply codes reflect both positive and negative
outcomes, there have been some innovative uses made for the text
accompanying positive responses, e.g., prayer wheels[40].
Regardless, some of the more modern application protocols include a
language localization parameter for the diagnostic text.
Finally, since the introduction of reply codes in 1981, two
unresolved criticisms have been raised:
o a reply code is used both to signal the outcome of an operation
and a change in the application protocol's state; and,
o a reply code doesn't specify whether the associated textual
diagnostic is destined for the end-user, administrator, or
programmer.
3.4 Multiplexing
Few application protocols today allow independent parallel exchanges
over the same connection. In fact, the more widely implemented
approach is to allow pipelining, e.g., command pipelining[16] in
SMTP or persistent connections in HTTP 1.1. Pipelining allows a
client to make multiple requests of a server, but requires the
requests to be processed serially. (Note that a protocol needs to
explicitly provide support for pipelining, since, without explicit
guidance, many implementors produce systems that don't handle
pipelining properly; typically, an error in a request causes
Rose Expires December 28, 2000 [Page 10]
Internet-Draft On the Design of Application Protocols June 2000
subsequent requests in the pipeline to be discarded).
Pipelining is a powerful method for reducing network latency. For
example, without persistent connections, HTTP's framing mechanism is
really closer to connection-blasting than octet-counting, and it
enjoys the same latency and efficiency problems.
In addition to reducing network latency (the pipelining effect),
parallelism also reduces server latency by allowing multiple
requests to be processed by multi-threaded implementations. Note
that if you allow any form of asynchronous exchange, then support
for parallelism is also required, because exchanges aren't
necessarily occurring under the synchronous direction of a single
peer.
Unfortunately, when you allow parallelism, you also need a flow
control mechanism to avoid starvation and deadlock. Otherwise, a
single set of exchanges can monopolize the bandwidth provided by the
transport layer. Further, if a peer is resource-starved, then it may
not have enough buffers to receive a message and deadlock results.
Flow control is typically implemented at the transport layer. For
example, TCP uses sequence numbers and a sliding window: each
receiver manages a sliding window that indicates the number of data
octets that may be transmitted before receiving further permission.
However, it's now time for the third shoe of multiplexing to drop:
segmentation. If you do flow control then you also need a
segmentation mechanism to fragment messages into smaller pieces
before sending and then re-assemble them as they're received.
All three of the multiplexing issues: parallelism, flow control, and
segmentation have an impact on how the protocol does framing. Before
we defined framing as "how to tell the beginning and end of each
message" -- in addition, we need to be able to identify independent
messages, send messages only when flow control allows us to, and
segment them if they're larger than the available window (or too
large for comfort).
Segmentation impacts framing in another way -- it relaxes the
octet-counting requirement that you need to know the length of the
whole message before sending it. With segmentation, you can start
sending segments before the whole message is available. In HTTP 1.1
you can "chunk" (segment) data to get this advantage.
3.5 User Authentication
Perhaps for historical (or hysterical) reasons, most application
protocols don't do authentication. That is, they don't authenticate
the identity of the peers on the connection or the authenticity of
Rose Expires December 28, 2000 [Page 11]
Internet-Draft On the Design of Application Protocols June 2000
the messages being exchanged. Or, if authentication is done, it is
domain-specific for each protocol. For example, FTP and HTTP use
entirely different models and mechanisms for authenticating the
initiator of a connection. (Independent of mainstream HTTP, there is
a little-used variant[17] that authenticates the messages it
exchanges.)
A few years ago, SASL[18] (the Simple Authentication and Security
Layer) was developed to provide a framework for authenticating
protocol peers. SASL let's you describe how an authentication
mechanism works, e.g., an OTP[19] (One-Time Password) exchange. It's
then up to each protocol designer to specify how SASL exchanges are
conveyed by the protocol. For example, [20] explains how SASL works
with SMTP.
A notable exception to the SASL bandwagon is HTTP, which defines its
own authentication mechanisms[21]. There is little reason why SASL
couldn't be introduced to HTTP, although to avoid race-conditions
with the use of OTP, the persistent connection mechanism of HTTP 1.1
must be used.
SASL has an interesting feature in that in addition to explicit
protocol exchanges to authenticate identity, it can also use
implicit information provided from the layer below. For example, if
the connection is running over IPsec[22], then the credentials of
each peer are known and verified when the TCP connection is
established.
3.6 Transport Security
HTTP is the first widely used protocol to make use of transport
security to encrypt the data sent on the connection. The current
version of this mechanism, TLS[23], is also available for SMTP and
other application protocols such as ACAP[24] (the Application
Configuration Access Protocol).
The key difference between the original mechanism and TLS, is one of
provisioning. In the initial approach, a world-wide web server would
listen on two ports, one for plaintext traffic and the other for
secured traffic; in contrast, a server implementing an application
protocol that is TLS-enabled listens on a single port for plaintext
traffic; once a connection is established, the use of TLS is
negotiated by the peers.
Rose Expires December 28, 2000 [Page 12]
Internet-Draft On the Design of Application Protocols June 2000
3.7 Let's Recap
Let's briefly compare the properties of the three main
connection-oriented application protocols in use today:
Mechanism SMTP FTP HTTP
------------------- ---------- --------- -------------
Framing Stuffing Blasting Counting
Encoding 822-style Binary MIME
Error Reporting 3-digit 3-digit 3-digit
Multiplexing pipelining none persistent
and chunky
User Authentication SASL user/pass user/pass
Transport Security TLS none TLS (nee SSL)
Note that the username/password mechanisms used by FTP and HTTP are
entirely different with one exception: both can be termed a
"username/password" mechanism.
These three choices are broadly representative: as more protocols
are considered, the patterns are reinforced. For example, POP[25]
uses octet-stuffing, but IMAP uses octet-counting, and so on.
Rose Expires December 28, 2000 [Page 13]
Internet-Draft On the Design of Application Protocols June 2000
4. Protocol Properties
When we design an application protocol, there are a few properties
that we should keep an eye on.
4.1 Scalability
A well-designed protocol is scalable.
Because few application protocols support multiplexing, a common
trick is for a program to open multiple simultaneous connections to
a single destination. The theory is that this reduces latency and
increases throughput. The reality is that both the transport layer
and the server view each connection as an independent instance of
the application protocol, and this causes problems.
In terms of the transport layer, TCP uses adaptive algorithms to
efficiently transmit data as networks conditions change. But what
TCP learns is limited to each connection. So, if you have multiple
TCP connections, you have to go through the same learning process
multiple times -- even if you're going to the same host. Not only
does this introduce unnecessary traffic spikes into the network,
because TCP uses a slow-start algorithm when establishing a
connection, the program still sees additional latency. To deal with
the fact that a lack of multiplexing in application protocols causes
implementors to make sloppy use of the transport layer, network
protocols are now provisioned with increasing sophistication, e.g.,
RED[26]. Further, suggestions are also being considered for
modification of TCP implementations to reduce concurrent learning,
e.g., [27].
In terms of the server, each incoming connection must be dispatched
and (probably) authenticated against the same resources.
Consequently, server overhead increases based on the number of
connections established, rather than the number of remote users. The
same issues of fairness arise: it's much harder for servers to
allocate resources on a per-user basis, when a user can cause an
arbitrary number of connections to pound on the server.
Another important aspect of scalability to consider is the relative
numbers of clients and servers. (This is true even in the
peer-to-peer model, where a peer can act both in the client and
server role.) Typically, there are many more client peers than
server peers. In this case, functional requirements should be
shifted from the servers onto the clients. The reason is that a
server is likely to be interacting with multiple clients and this
functional shift makes it easier to scale.
Rose Expires December 28, 2000 [Page 14]
Internet-Draft On the Design of Application Protocols June 2000
4.2 Efficiency
A well-designed protocol is efficient.
For example, although a compelling argument can be made than
octet-stuffing leads to more elegant implementations than
octet-counting, experience shows that octet-counting consumes far
fewer cycles.
Regrettably, we sometimes have to compromise efficiency in order to
satisfy other properties. For example, 822 (and MIME) use textual
headers. We could certainly define a more efficient representation
for the headers if we were willing to limit the header names and
values that could be used. In this case, extensibility is viewed as
more important than efficiency. Of course, if we were designing a
network protocol instead of an application protocol, then we'd make
the trade-offs using a razor with a different edge.
4.3 Simplicity
A well-designed protocol is simple.
Here's a good rule of thumb: a poorly-designed application protocol
is one in which it is equally as "challenging" to do something basic
as it is to do something complex. Easy things should be easy to do
and hard things should be harder to do. The reason is simple: the
pain should be proportional to the gain.
Another rule of thumb is that if an application protocol has two
ways of doing the exact same thing, then there's a problem somewhere
in the architecture underlying the design of the application
protocol.
Hopefully, simple doesn't mean simple-minded: something that's
well-designed accommodates everything in the problem domain, even
the troublesome things at the edges. What makes the design simple is
that it does this in a consistent fashion. Typically, this leads to
an elegant design.
4.4 Extensibility
A well-designed protocol is extensible.
As clever as application protocol designers are, there are likely to
be unforeseen problems that the application protocol will be asked
to solve. So, it's important to provide the hooks that can be used
to add functionality or customize behavior. This means that the
protocol is evolutionary, and there must be a way for
implementations reflecting different steps in the evolutionary path
Rose Expires December 28, 2000 [Page 15]
Internet-Draft On the Design of Application Protocols June 2000
to negotiate which extensions will be used.
But, it's important to avoid falling into the extensibility trap:
the hooks provided should not be targeted at half-baked future
requirements. Above all, the hooks should be simple.
Of course good design goes a long way towards minimizing the need
for extensibility. For example, although SMTP initially didn't have
an extension framework, it was only after ten years of experience
that its excellent design was altered. In contrast, a
poorly-designed protocol such as Telnet[28] can't function without
being built around the notion of extensions.
Rose Expires December 28, 2000 [Page 16]
Internet-Draft On the Design of Application Protocols June 2000
5. The BXXP Framework
Finally, we get to the money shot: here's what we did.
We defined an application protocol framework called BXXP (the Blocks
eXtensible eXchange Protocol). The reason it's a "framework" instead
of an application protocol is that we provide all the mechanisms
discussed earlier without actually specifying the kind of messages
that get exchanged. So, when someone else needs an application
protocol that requires connection-oriented, asynchronous
request/response interactions, they can start with BXXP. It's then
their responsibility to define the last 10% of the application
protocol, the part that does, as we say, "the useful work".
So, what does BXXP look like?
Mechanism BXXP
------------------- ----------------------------------------
Framing Counting, with a trailer
Encoding MIME, defaulting to text/xml
Error Reporting 3-digit and localized textual diagnostic
Multiplexing independent request/response streams
User Authentication SASL
Transport Security SASL or TLS
5.1 Framing and Encoding
Framing in BXXP looks a lot like SMTP or HTTP: there's a command
line that identifies the beginning of the frame, then there's a MIME
object (headers and body). Unlike SMTP, BXXP uses octet-counting,
but unlike HTTP, the command line is where you find the size of the
payload. Finally, there's a trailer after the MIME object to aid in
detecting framing errors.
Actually, the command line for BXXP has a lot of information, it
tells you:
o whether this frame contains a request or response;
o whether there's more to the message than just what's in this
frame (a continuation flag);
o how to distinguish the message contained in this frame from other
messages (a serial number);
Rose Expires December 28, 2000 [Page 17]
Internet-Draft On the Design of Application Protocols June 2000
o where the payload occurs in the sliding window (a sequence
number) along with how many octets are in the payload of this
frame; and,
o which part of the system should get the message (for requests) or
whether this is a positive or negative response.
(The command line is textual and ends in a CR-LF pair, and the
arguments are separated by a space.)
Since you need to know all this stuff to process a frame, we put it
all in one easy to parse location. You could probably devise a more
efficient encoding, but the command line is a very small part of the
frame, so you wouldn't get much bounce from optimizing it. Further,
because framing is at the heart of BXXP, the frame format has
several consistency checks that catch the majority of programming
errors. (The combination of a sequence number, an octet count, and a
trailer allows for very robust error detection.)
Another trick is in the headers: because the command line contains
all the framing information, the headers may contain minimal MIME
information (such as Content-Type). Usually, however, the headers
are empty. That's because the BXXP default payload is XML[29].
(Actually, a "Content-Type: text/xml" with binary transfer encoding).
We chose XML as the default because it provides a simple mechanism
for nested, textual representations. (Alas, the 822-style encoding
doesn't easily support nesting.) By design, XML's nature isn't
optimized for compact representations. That's okay because we're
focusing on loosely-coupled systems and besides there are efficient
XML parsers available. Further, there's a fair amount of anecdotal
experience -- and we'll stress the word "anecdotal" -- that if you
have any kind of compression (either at the link-layer or during
encryption), then XML encodings squeeze down nicely.
Even so, use of XML is probably the most controversial part of BXXP.
After all, there are more efficient representations around. We
agree, but the real issue isn't efficiency, it's ease of use: there
are a lot of people who grok the XML thing and there are a lot of
XML tools out there. The pain of recreating this social
infrastructure far outweighs any benefits of devising a new
representation. So, if the "make" option is too expensive, is there
something else we can "buy" besides XML? Well, there's ASN.1/BER
(just kidding).
In the early days of the SNMP[30], which does use ASN.1, the same
issues arose. In the end, the working group agreed that the use of
ASN.1 for SNMP was axiomatic, but not because anyone thought that
ASN.1 was the most efficient, or the easiest to explain, or even
Rose Expires December 28, 2000 [Page 18]
Internet-Draft On the Design of Application Protocols June 2000
well liked. ASN.1 was given axiomatic status because the working
group decided it was not going to spend the next three years
explaining an alternative encoding scheme to the developer community.
So -- and we apologize for appealing to dogma -- use of XML as the
favored encoding scheme in BXXP is axiomatic.
5.2 Error Reporting
We use 3-digit error codes, with a localized textual diagnostic.
(Each peer specifies a preferred ordering of languages.)
In addition, the response message to a request is flagged as either
positive or negative. This makes it easy to signal success or
failure and allow the receiving peer some freedom in the amount of
parsing it wants to do on failure.
5.3 Multiplexing
Despite the lessons of SMTP and HTTP, there isn't a lot of field
experience to rely on when designing the multiplexing features of
BXXP. (Actually, there were several efforts in 1998 related to
application layer framing, e.g., [31], but none appear to have
achieved orbit.)
So, here's what we did: frames are exchanged in the context of a
"channel". Each channel has an associated "profile" that defines the
syntax and semantics of the messages exchanged over a channel.
Channels provide both an extensibility mechanism for BXXP and the
basis for multiplexing. Remember the last parameter in the command
line of a BXXP frame? The "part of the system" that gets the message
is identified by a channel number.
A profile is defined according to a "Profile Registration" template.
The template defines how the profile is identified (using a
URI[32]), what kind of messages get exchanged during channel
creation, what kind of messages get sent in requests and responses,
along with the syntax and semantics of those messages. When you
create a channel, you identify a profile and maybe piggyback your
first request. If the channel is successfully created, you get back
a positive response; otherwise, you get back a negative response
explaining why.
Perhaps the easiest way to see how channels provide an extensibility
mechanism is to consider what happens when a connection is
established. The BXXP peer that accepted the connection sends a
greeting on channel zero identifying the profiles that it supports.
(Channel 0 is used for channel management -- it's automatically
Rose Expires December 28, 2000 [Page 19]
Internet-Draft On the Design of Application Protocols June 2000
created when a connection is opened.) If you want transport
security, the very first thing you do is to create a channel that
negotiates transport security, and, once the channel is created, you
tell it to do its thing. Next, if you want to authenticate, you
create a channel that performs user authentication, and, once the
channel is created, you tell it to get busy. At this point, you
create one or more channels for data exchange. This process is
called "tuning"; once you've tuned the connection, you start using
the data exchange channels to do "the useful work".
The first channel that's successfully started has a trick associated
with it: when you ask to start the channel, you're allowed to
specify a "service name" that goes with it. This allows a server
with multiple configurations to select one based on the client's
suggestion. (A useful analogy is HTTP 1.1's "Host:" header.) If the
server accepts the "service name", then this configuration is used
for the rest of the connection.
To allow parallelism, BXXP allows you to use multiple channels
simultaneously. Each channel processes requests serially, but there
are no constraints on the processing order for different channels.
So, in a multi-threaded implementation, each channel maps to its own
thread.
This is the most general case, of course. For one reason or another,
an implementor may not be able to support this. So, BXXP allows for
both positive and negative responses when a request is made. So, if
you want the classic client/server model, the client program should
simply reject any requests made by the server. This effectively
throttles any asynchronous messages from the server.
Of course, we now need to provide mechanisms for segmentation and
flow control. For the former, we just put a "continuation" or "more
to come" flag in the command line for the frame. For the latter, we
introduced the notion of a "transport mapping".
What this means is that BXXP doesn't directly define how it sits of
top of TCP. Instead, it lists a bunch of requirements for how a
transport service needs to support a BXXP session. Then, in a
separate document, we defined how you can use TCP to meet these
requirements.
This second document pretty much says "use TCP directly", except
that it introduces a flow control mechanism for multiplexing
channels over a single TCP connection. The mechanism we use is the
same one used by TCP (sequence numbers and a sliding window). It's
proven, and can be trivially implemented by a minimal implementation
of BXXP.
Rose Expires December 28, 2000 [Page 20]
Internet-Draft On the Design of Application Protocols June 2000
The introduction of flow control is a burden from an implementation
perspective -- although TCP's mechanism is conceptually simple, an
implementor must take great care. For example, issues such as
priorities, queue management, and the like should be addressed.
Regardless, we feel that the benefits of allowing parallelism for
intra-application streams is worth it. (Besides, our belief is that
few application implementors will actually code the BXXP framework
directly -- rather, we expect them to use third-party packages that
implement BXXP.)
5.4 User Authentication
We use SASL. If you successfully authenticate using a channel, then
there is a single user identity for each peer on that connection
(i.e., authentication is per-connection, not per-channel). This
design decision mandates that each connection correspond to a single
user regardless of how many channels are open on that connection.
One reason why this is important is that it allows service
provisioning, such as quality of service (e.g., as in [33]) to be
done on a per-user granularity.
5.5 Transport Security
We use SASL and TLS. If you successfully complete a transport
security negotiation using a channel, then all traffic on that
connection is secured (i.e., confidentiality is per-connection, not
per-channel, just like authentication).
We defined a BXXP profile that's used to start the TLS engine.
Rose Expires December 28, 2000 [Page 21]
Internet-Draft On the Design of Application Protocols June 2000
5.6 Things We Left Out
We purposefully excluded two things that are common to most
application protocols: naming and authorization.
Naming was excluded from the framework because, outside of URIs,
there isn't a commonly accepted framework for naming things. To our
view, this remains a domain-specific problem for each application
protocol. Maybe URIs are appropriate in the context of a
particularly problem domain, maybe not. So, when an application
protocol designer defines their own profile to do "the useful work",
they'll have to deal with naming issues themselves. BXXP provides a
mechanism for identifying profiles and binding them to channels.
It's up to you to define the profile and use the channel.
Similarly, authorization was explicitly excluded from the framework.
Every approach to authorization we've seen uses names to identify
principals (i.e., targets and subjects), so if a framework doesn't
include naming, it can't very well include authorization.
Of course, application protocols do have to deal with naming and
authorization -- those are two of the issues addressed by the
applications protocol designer when defining a profile for use with
BXXP.
Rose Expires December 28, 2000 [Page 22]
Internet-Draft On the Design of Application Protocols June 2000
6. Current Status
So, how do you go about using BXXP?
First, get the framework specification[35] and read it. Next, define
your own profile. Finally, get one of the open source SDKs (in Perl
or Java) and start coding.
The BXXP specification defines several profiles itself: a channel
management profile, a family of profiles for SASL, and a transport
layer security profile. These provide good examples. Of course,
we've been using BXXP internally for a year and a half now, so if
you want to look at a rather detailed profile definition, check out
the Blocks Simple Exchange[34] profile. It addresses the issue of
naming for its application domain, and, in doing so, opens the door
for authorization.
Rose Expires December 28, 2000 [Page 23]
Internet-Draft On the Design of Application Protocols June 2000
7. To Be Determined...
Since the initial publication of BXXP, we've gotten some pretty good
feedback. As a result, a number of changes and clarifications were
incorporated. There are, however, a few open issues that we'll need
to decide soon:
closing channels: At present, channel management doesn't allow you
to close a channel -- we never envisioned the need given that
BXXP allows 127 channels to be open by each peer. However, the
question comes up a lot. We can either add this functionality to
channel management, increase the number of channels, or do
nothing.
reply code limitations: At present, BXXP has a generic "error"
element used to convey a reply code and textual diagnostic. This
element doesn't address either of the two issues currently raised
with reply codes: overloading of the first digit and diagnostic
targeting.
textual debugging: At present, it's hard to debug BXXP sessions by
hand because the framing protocol includes an octet count. One
suggestion is to add an optional feature to BXXP that uses the
trailer as the final delimiter, at the cost of reducing both the
efficiency and robustness of the framing protocol.
transport mappings: At present, only one transport mapping for BXXP
has been defined (the mapping of a BXXP session onto a single TCP
connection). Other mappings are possible, most notably onto
UDP[37] or onto SCTP[38].
In mapping the BXXP framework onto UDP, additional mechanisms
must be added, e.g., achieving reliability through
retransmission.
In mapping the BXXP framework onto SCTP, BXXP can achieve
multiplexing without having to provide a mechanism for flow
control. The reason is that SCTP explicitly separates reliabilty
from flow control (in TCP they are bundled together). In essence,
each BXXP channel would have a separate window managed by SCTP,
and yet retain congestion avoidance characteristics across the
entire BXXP connection. Accordingly, a future mapping of BXXP
onto SCTP is simpler than the TCP mapping defined in [36].
the name: A lot of folks on slashdot think that we should call this
BEEP. We have to admit, there are a lot of comic possibilities...
If you have an opinion on any of these issues, let us know!
Rose Expires December 28, 2000 [Page 24]
Internet-Draft On the Design of Application Protocols June 2000
References
[1] Postel, J., "Simple Mail Transfer Protocol", RFC 821, STD 10,
Aug 1982.
[2] Rose, M.T. and C. Malamud, "Blocks: Architectural Precepts",
draft-mrose-blocks-architecture-01 (work in progress), March
2000.
[3] Postel, J. and J.K. Reynolds, "File Transfer Protocol", RFC
959, STD 9, Oct 1985.
[4] Berners-Lee, T., Fielding, R. and H. Frystyk, "Hypertext
Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996.
[5] Herriot, R., "Internet Printing Protocol/1.0: Encoding and
Transport", RFC 2565, April 1999.
[6] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part One: Format of Internet Message
Bodies", RFC 2045, November 1996.
[7] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter,
L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol
-- HTTP/1.1", RFC 2616, June 1999.
[8] Postel, J., "Transmission Control Protocol", RFC 793, STD 7,
Sep 1981.
[9] Mockapetris, P.V., "Domain names - concepts and facilities",
RFC 1034, STD 13, Nov 1987.
[10] Microsystems, Sun, "NFS: Network File System Protocol
specification", RFC 1094, Mar 1989.
[11] Srisuresh, P. and M. Holdrege, "IP Network Address Translator
(NAT) Terminology and Considerations", RFC 2663, August 1999.
[12] Crocker, D., "Standard for the format of ARPA Internet text
messages", RFC 822, STD 11, Aug 1982.
[13] Berners-Lee, T. and D. Connolly, "Hypertext Markup Language -
2.0", RFC 1866, November 1995.
[14] Freed, N., "SMTP Service Extension for Returning Enhanced
Error Codes", RFC 2034, October 1996.
[15] Myers, J., "IMAP4 Authentication Mechanisms", RFC 1731,
December 1994.
Rose Expires December 28, 2000 [Page 25]
Internet-Draft On the Design of Application Protocols June 2000
[16] Freed, N., "SMTP Service Extension for Command Pipelining",
RFC 2197, September 1997.
[17] Rescorla, E. and A. Schiffman, "The Secure HyperText Transfer
Protocol", RFC 2660, August 1999.
[18] Myers, J.G., "Simple Authentication and Security Layer
(SASL)", RFC 2222, October 1997.
[19] Newman, C., "The One-Time-Password SASL Mechanism", RFC 2444,
October 1998.
[20] Myers, J., "SMTP Service Extension for Authentication", RFC
2554, March 1999.
[21] Franks, J., Hallam-Baker, P., Hostetler, J., Lawrence, S.,
Leach, P., Luotonen, A. and L. Stewart, "HTTP Authentication:
Basic and Digest Access Authentication", RFC 2617, June 1999.
[22] Kent, S. and R. Atkinson, "Security Architecture for the
Internet Protocol", RFC 2401, November 1998.
[23] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC
2246, January 1999.
[24] Newman, C. and J. G. Myers, "ACAP -- Application Configuration
Access Protocol", RFC 2244, November 1997.
[25] Myers, J. and M. Rose, "Post Office Protocol - Version 3", RFC
1939, STD 53, May 1996.
[26] Braden, B., Clark, D.D., Crowcroft, J., Davie, B., Deering,
S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
Partridge, C., Peterson, L., Ramakrishnan, K.K., Shenker, S.,
Wroclawski, J. and L. Zhang, "Recommendations on Queue
Management and Congestion Avoidance in the Internet", RFC
2309, April 1998.
[27] Touch, J., "TCP Control Block Interdependence", RFC 2140,
April 1997.
[28] Postel, J. and J.K. Reynolds, "Telnet Protocol Specification",
RFC 854, May 1983.
[29] World Wide Web Consortium, "Extensible Markup Language (XML)
1.0", W3C XML, February 1998,
<http://www.w3.org/TR/1998/REC-xml-19980210>.
[30] Case, J.D., Fedor, M., Schoffstall, M.L. and C. Davin, "Simple
Rose Expires December 28, 2000 [Page 26]
Internet-Draft On the Design of Application Protocols June 2000
Network Management Protocol (SNMP)", RFC 1157, STD 15, May
1990.
[31] World Wide Web Consortium, "SMUX Protocol Specification",
Working Draft, July 1998,
<http://www.w3.org/TR/1998/WD-mux-19980710>.
[32] Berners-Lee, T., Fielding, R.T. and L. Masinter, "Uniform
Resource Identifiers (URI): Generic Syntax", RFC 2396, August
1998.
[33] Waitzman, D., "IP over Avian Carriers with Quality of
Service", RFC 2549, April 1999.
[34] Rose, M.T., "The Blocks Simple Exchange Profile",
draft-mrose-blocks-exchange-01 (work in progress), April 2000.
[35] Rose, M.T., "The Blocks eXtensible eXchange Protocol
Framework", draft-mrose-bxxp-framework-00 (work in progress),
June 2000.
[36] Rose, M.T., "Mapping the BXXP Framework onto TCP",
draft-mrose-bxxp-tcpmapping-00 (work in progress), June 2000.
[37] Postel, J., "User Datagram Protocol", RFC 768, STD 6, Aug 1980.
[38] Stewart, R.R., Xie, Q., Morneault, K., Sharp, C.,
Schwarzbauer, H.J., Taylor, T., Rytina, I., Kalla, M., Zhang,
L. and V. Paxson, "Stream Control Transmission Control
Protocol", draft-ietf-sigtran-sctp-10 (work in progress),
April 2000.
[39] mailto:ddc@lcs.mit.edu
[40] http://mappa.mundi.net/cartography/Wheel/
Author's Address
Marshall T. Rose
Invisible Worlds, Inc.
1179 North McDowell Boulevard
Petaluma, CA 94954-6559
US
Phone: +1 707 789 3700
EMail: mrose@invisible.net
URI: http://invisible.net/
Rose Expires December 28, 2000 [Page 27]
Internet-Draft On the Design of Application Protocols June 2000
Full Copyright Statement
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Invisible Worlds expressly disclaims any and all warranties
regarding this contribution including any warranty that (a) this
contribution does not violate the rights of others, (b) the owners,
if any, of other rights in this contribution have been informed of
the rights and permissions granted to IETF herein, and (c) any
required authorizations from such owners have been obtained. This
document and the information contained herein is provided on an "AS
IS" basis and INVISIBLE WORLDS DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
IN NO EVENT WILL INVISIBLE WORLDS BE LIABLE TO ANY OTHER PARTY
INCLUDING THE IETF AND ITS MEMBERS FOR THE COST OF PROCURING
SUBSTITUTE GOODS OR SERVICES, LOST PROFITS, LOSS OF USE, LOSS OF
DATA, OR ANY INCIDENTAL, CONSEQUENTIAL, INDIRECT, OR SPECIAL DAMAGES
WHETHER UNDER CONTRACT, TORT, WARRANTY, OR OTHERWISE, ARISING IN ANY
WAY OUT OF THIS OR ANY OTHER AGREEMENT RELATING TO THIS DOCUMENT,
WHETHER OR NOT SUCH PARTY HAD ADVANCE NOTICE OF THE POSSIBILITY OF
SUCH DAMAGES.
Rose Expires December 28, 2000 [Page 28]
Internet-Draft On the Design of Application Protocols June 2000
Acknowledgement
Funding for the RFC editor function is currently provided by the
Internet Society.
Rose Expires December 28, 2000 [Page 29]