Internet-Draft K. Evans
Expires in 6 months J. Klein
Tandem Computers
J. Lyon
Microsoft
July 4 1997
Transaction Internet Protocol - Requirements
<draft-evans-tip-functions-00.txt>
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
To learn the current status of any Internet-Draft, please check the
"1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast).
Distribution of this document is unlimited. Please send comments to
the authors at <JimLyon@Microsoft.Com>, <Evans_Keith@Tandem.Com>,
or <Klein_Johannes@Tandem.Com>.
Abstract
This document describes the purpose (usage scenarios) and
requirements for the Transaction Internet Protocol [1]. It is
intended to help qualify the necessary features and functions of
the protocol.
Introduction
Transactions are a very useful programming paradigm, greatly
simplifying the writing of distributed applications. When
transactions are employed, no matter how many distributed
application components participate in a particular unit-of-work,
the number of possible outcomes is reduced to only two; that is,
either all of the work completed successfully, or none of it did
(this characteristic is known as atomicity). Applications
programming is therefore much less complex since the programmer
does not have to deal with a multitude of possible failure
scenarios. Typically, transaction semantics are provided by some
underlying system infrastructure (usually in the form of products
such as Transaction Processing Monitors, and/or Databases). This
infrastructure deals with failures, and performs the necessary
recovery actions to guarantee the property of atomicity. The use
of transactions enables the development of reliable distributed
applications which would otherwise be difficult, if not impossible.
A key technology required to support distributed transactions is
the two-phase commit protocol (2-pc). 2-pc protocols have been used
in commercial Transaction Processing (TP) systems for many years,
and are well understood (e.g. the LU6.2 2-pc (syncpoint) protocol
was first implemented more than 12 years ago). Today a number of
different 2-pc protocols are supported by a variety of TP monitor
and database products. 2-pc is used between the components
participating in a distributed unit-of-work (transaction) to
ensure agreement by all parties regarding the outcome of that work
(regardless of any failure).
Today both standard and proprietary 2-pc protocols exist. These
protocols typically employ a "one-pipe" model. That is, the
transaction and application protocols are tightly-integrated,
executing over the same communications channel. An application may
use only the particular communications mechanism associated with
the transaction protocol. The older standard protocols (OSI TP,
LU6.2) are complex, with a large footprint and extensive
configuration and administration requirements. For these reasons
they are not very widely deployed. A new protocol, CORBA OTS, still
adheres to the one-pipe model, and is tightly coupled with a
particular application environment. The net of all this is
restricted application flexibility and interoperability if
transactions are to be used. Applications may wish to use a number
of communications protocols for which there are no transactional
variants (e.g. HTTP), and be deployed in very heterogeneous
application environments.
In summary, transactions greatly simplify the programming of
distributed applications, and the 2-pc protocol is a key
transactional technology. Current 2-pc protocols only offer
transaction semantics to a limited set of applications, operating
within a special-purpose (complex, homogeneous) infrastructure,
using a particular set of intercommunication protocols. The
restrictions thus imposed by current 2-pc protocols limits the
widespread use of the transaction paradigm, thereby inhibiting the
development of new distributed business applications.
(See [2] for more information re transactions, atomicity, and
two-phase commit protocols in general.)
The Transaction Internet Protocol (TIP)
TIP is a 2-pc protocol which is intended to provide ubiquitous
distributed transaction support, in a heterogeneous (networked)
environment. TIP removes the restrictions of current 2-pc
protocols and enables the development of new distributed business
applications.
This goal is achieved primarily by satisfying two key
requirements:
1. Keep the protocol simple (yet functionally sufficient). If the
protocol is complex it will not be widely deployed or quickly
adopted. Simplicity also means suitability to a wide range of
application environments.
2. Enable the protocol to be used with any applications
communications protocol (e.g. HTTP). This ensures heterogeneous
environments can participate in distributed work.
TIP does not reinvent the 2-pc protocol itself, the well-known
presumed-abort 2-pc protocol is used as a basis. Rather the novelty
and utility of TIP is in its separation from the application
communications protocol (the two-pipe model).
Scope
Note that TIP does not describe how business transactions or
electronic commerce are to be conducted on the internet, it
specifies only the 2-pc transaction protocol (which is an aid in
the development of such applications). TIP does not specify
Application Programming Interfaces.
Anticipated Usage of TIP
As described above, transactions are a very useful tool in
simplifying the programming of distributed applications. TIP is
therefore targeted at any application that involves distributed
work. Such applications may comprise components executing within a
single system, across a corporate intranet, across the internet, or
any other distributed system configuration. The application
may be of "enterprise" class (requiring high-levels of performance
and availability), or be less demanding. TIP is intended to
be generally applicable, meeting the requirements of any
application type which would benefit from the provision of
transaction semantics.
TIP Compliant Systems
There are two classes of TIP compliant Transaction Manager system:
1. Client-only systems. Those which provide an application
interface to demarcate TIP transactions, but which do not offer
access to local recoverable resources. Such a lightweight
implementation is useful for systems which host client
applications only (e.g. desktop machines). Such client systems
may be unreliable, and are not appropriate as transaction
coordinators (their unavailability might cause resources on
other transaction participant systems to remain locked and
unavailable). These so-called "volatile client" systems
therefore delegate the responsibility to coordinate the
transaction (and recover from failures), to other "full"
(server) TIP system implementations. For these lightweight
systems, only the TIP IDENTIFY, BEGIN, COMMIT, and ABORT
commands are needed; no transaction log is required.
2. Server systems. Those which offer the above support, plus TIP
transaction coordination and recovery services. These systems
may also provide access to recoverable resources
(e.g. relational databases). Server systems support all TIP
commands, and provide a recoverable transaction log.
A TIP compliant Transaction Manager (TM), will also supply
application programming interfaces to demarcate transactions
(e.g. the X/Open TX interface [3]), plus commands to generate TIP
URLs, to PUSH/PULL TIP transactions, and to set the current TIP
transaction context. TIP support can be added to TMs with existing
APIs and 2-pc protocols, and transactions may comprise both
proprietary and TIP transaction branches (it is assumed existing TM
implementations will provide "TIP gateway" facilities which will
coordinate between TIP and other transaction protocols).
Relationship to the X/Open DTP Model
The X/Open Distributed Transaction Processing (DTP) Model [4]
defines four components: 1) Application Program (AP),
2) Transaction Manager (TM), 3) Resource Manager (RM),
and 4) Communications Resource Manager (CRM). In this model, TIP
defines a TM to TM interoperability protocol, which is independent
of application communications (there is no such equivalent protocol
specified by X/Open, where all transaction and application
communication occurs between CRMs (the one-pipe model)).
Programmatic interfaces between the AP and TM/RM are unaffected by,
and may be used with TIP. The TM to RM interaction is defined via
the X/Open XA interface specification [5]. TIP is compatible with
XA, and a TIP transaction may comprise applications accessing
multiple RMs where the XA interface is being used to coordinate
the RM transaction branches.
Example TIP Usage Scenario
It is expected that a typical internet usage of TIP will involve
applications using the agency model. In this model, the client node
itself is not directly involved in the TIP protocol at all, and
does not need the services of a local TIP TM. Instead, an agency
(server) application handles the dialogue with the client, and is
responsible for the coordination of the TIP transaction. The agency
works with other service providers to deliver the service to the
client. e.g. as a Travel Agency acts as an intermediate between
airlines/hotels/etc and the customer. A big benefit of this model
is that the agency is trusted by the service providers, and there
are fewer such agencies (compared to user clients), so issues of
security and performance are reduced.
Consider a Travel Agency example. A client running a web browser
on a network PC accesses the Travel Agency web page. Via pages
served up by the agency (which may in turn may be constructed from
pages provided by the airline and hotel servers), the client
creates an itinerary involving flights and hotel choices. Finally,
the client clicks the "make reservation" button. At this point the
following sequence of events occurs (user-written application code
is invoked by the various web servers, via any of the standard or
proprietary techniques available (e.g. CGI)):
1. The travel agency begins a local transaction, and gets a TIP URL
for this transaction (both of these functions are performed
using the API of the local TM). The TIP URL contains the
listening endpoint IP address of the local TM and the
transaction identifier of the current transaction.
2. The travel agency application sends a request to the airline
server (via some protocol (e.g. HTTP) specified via some URL),
requesting the "book_flight" service, passing the flights
selected by the client, and the TIP URL (obtained in 1. above).
3. The request is received by the airline server which invokes the
book_flight application. This application retrieves the TIP URL
from the input data, and passes this on a "tip_pull" API request
to its local TM. The tip_pull function causes the following to
occur:
a. the local TM creates a transaction branch (under which the
work will be performed),
b. if a TIP connection does not already exist to the superior
(travel agency) TM (as identified via the IP address passed
in the TIP URL), one is created and an IDENTIFY exchange
occurs (if multiplexing is to be used on the connection, this
is followed by a MULTIPLEX exchange),
c. a PULL command is sent to the superior TM,
d. in response to the PULL, the superior TM associates the
subordinate (airline) TM with the transaction (by associating
the connection with the transaction),
e. the subordinate TM returns control to the book_flight
application, which is now executing in the context of the
newly created local transaction branch.
4. The book_flight application does its work (which may involve
access to a recoverable resource manager (e.g. an RDBMS), in
which case the local TM will associate the RM with the
transaction branch (via the XA interface or whatever)).
5. The book_flight application returns to the travel agency
application indicating success.
6. Steps 2-5 are then repeated with the hotel server "book_room"
application. At the conclusion of this, the superior TM has
registered two subordinate TMs as participants in the
transaction, there are TIP connections between the agency TM and
the airline and hotel TMs, and there are inflight transactions
at the airline and hotel servers. [Note that steps 2-5 and 6
could be performed in parallel.]
7. The travel agency application issues a "commit transaction"
request (using the API of the local TM). The local TM sends a
PREPARE command on the TIP connections to the airline and hotel
TMs (as these are registered as subordinate transaction
participants).
8. The TMs at the airline and hotel servers perform the
necessary steps to prepare their local recoverable resources
(e.g. by issuing xa_prepare requests). If successful, the
subordinate TMs change their TIP transaction state to Prepared,
and log recovery information (e.g. local and superior
transaction branch identifiers, and the IP address of the
superior TM). The subordinate TMs then send PREPARED commands to
the superior TM.
9. If both subordinates respond PREPARED, the superior TM logs that
the transaction is Committed, with recovery information
(e.g. local and subordinate transaction identifiers, and
subordinate TM IP addresses). The superior TM then sends COMMIT
commands on the two subordinate TIP connections.
10. The TMs at the airline and hotel servers perform the
necessary steps to commit their local recoverable resources
(e.g. by issuing xa_commit requests). The subordinate TMs
forget the transaction. The subordinate TMs then send COMITTED
commands to the superior TM.
11. The superior TM forgets the transaction. The TIP connections
between the superior and subordinate TMs return to Idle state
(not associated with any transaction). The superior TM returns
success to the travel agency application "commit transaction"
request.
12. The travel agency application returns "reservation made" to the
client.
TIP Transaction Recovery
Until the transaction reaches the Prepared state, any failure
results in the transaction being aborted. If an error occurs once
the transaction has reached the Prepared state, then transaction
recovery must be performed. Recovery behaviour is different for
superior and subordinate; the details depend upon the outcome of
the transaction (committed or aborted), and the precise point at
which failure occurs.
In the travel agency application for example, if the connection to
the hotel server fails before the COMMIT command has been received
by the hotel TM, then (once the connection is restored):
1. The superior (travel agency) TM sends a RECONNECT command
(passing the subordinate transaction identifier (recovered from
the transaction log if necessary)).
2. The subordinate (hotel) TM responds RECONNECTED (since it never
received the COMMIT command, and still has the transaction in
Prepared state (if the failure had occurred after the
subordinate had responded COMMITTED, then the subordinate would
have forgotten the transaction, and responded NOTRECONNECTED to
the RECONNECT command)).
3. The superior TM sends a COMMIT command. The subordinate TM
commits the transaction and responds COMMITTED. The transaction
is now resolved.
4. If the subordinate TM restores the connection to the superior TM
before receiving a RECONNECT command, then it may send a QUERY
command. In this case, the superior TM will respond
QUERIEDEXISTS, and the subordinate TM should wait for the
superior to send a RECONNECT command. If the transaction had
been aborted, then the superior may respond QUERIEDNOTFOUND, in
which case the subordinate should abort the transaction (note
that the superior is not obliged to send a RECONNECT command for
an aborted transaction (i.e. it could just forget the
transaction after sending ABORT and before receiving an ABORTED
response)).
Security
The means by which applications communicate and perform distributed
work are outside the scope of the TIP protocol. The mechanisms used
for authentication and authorisation of clients to access programs
and information on a particular system are part of the application
communications protocol and the application execution
infrastructure. Use of the TIP protocol does not affect these
considerations.
Security relates to the TIP protocol itself inasmuch that systems
require to protect themselves from the receipt of unauthorised TIP
commands, or the impersonation of a trusted partner TIP TM.
Probably the worst consequence of this is the possibility of
undetected data inconsistency resulting from violations of the TIP
commitment protocol (e.g. a COMMIT command is injected on a TIP
connection in place of an ABORT command). TIP implementations
concerned about this kind of attack can use the Secure Socket Layer
protocol [6] to restrict access to only trusted partners (i.e. to
control from which remote endpoints TIP transactions will be
accepted, and to verify that an end-point is genuine), and to
encrypt TIP commands.
TIP TM implementations will also likely provide local means to
time-out and abort transactions which have not completed within
some time period (thereby preventing unavailability of resources
due to malicious intent).
TIP Requirements
Most of these requirements stem from the primary objective of
making transactions a ubiquitous system service, available to all
application classes (much as TCP may be assumed to be available
everywhere). In general this requires imposing as few restrictions
regarding the use of TIP as possible (applications should not be
required to execute in some "special" environment in order to use
transactions), and keeping the protocol simple and efficient. This
enables the widespread implementation of TIP (it's cheap to do), on
a wide range of systems (it's cheap to run).
1. Application Communications Protocol Independence
The TIP protocol must be defined independently of the
communications protocol used for transferring application data,
to allow TIP usage in conjunction with any application protocol.
It must be possible for applications using arbitrary
communications protocols to begin, end, and propagate TIP
transactions.
This implies that the TIP protocol employ a 2-pipe model of
operation. This model requires the separation of application
communications and transaction coordination, into two discrete
communication channels (pipes). This separation enables the use
of the transaction coordination protocol (TIP), with any
application communications protocol (e.g. HTTP, ODBC, plain
TCP/UDP, etc).
2. Support for Transaction Semantics
The TIP protocol must provide the functionality of the de-facto
standard presumed-abort 2-pc protocol, to guarantee
transactional atomicity even in the event of failure. It should
provide a means to construct the transaction tree, as well as
provide commitment and recovery functions.
3. Application Transaction Propagation and Interoperability
In order to facilitate protocol independence, application
interoperability, and provide a means for TIP transaction
context propagation, a standard representation of the TIP
transaction context information is required (in the form of a
URL). This information must include the listening endpoint
address of the partner TIP TM, and transaction identifier
information.
4. Ease of Implementation
The TIP protocol must be simple to implement. It should support
only those features necessary to provide a useful, performant
2-pc protocol service. The protocol should not add complexity in
the form of extraneous optimisations.
5. Suitability for All Application Classes
The TIP protocol should be complete and robust enough not only
for electronic commerce on the web, but also for intranet
applications and for traditional TP applications spanning
heterogenous transaction manager environments. The protocol
should be performant and scaleable enough to meet the needs of
low to very high throughput applications.
- the TIP protocol should support the concept of client-only
transaction participants (useful for ultra-lightweight
implementations on low-end platforms)
- since some clients may be unreliable, TIP must provide support
for delegation of transaction coordination (to a more reliable
(trusted) node)
- the TIP protocol must scale between 1 and n (> 1) concurrent
transactions per TCP connection
- TIP commands should be able to be concatenated (pipelined)
- TIP should be compatible with the X/Open XA interface
6. Security
The TIP protocol must be compatible with existing security
mechanisms, potentially including encryption, firewalls, and
authorization mechanisms (e.g. SSL should be used to
authenticate the sender of a TIP command, and for encryption of
TIP commands). Nothing in the protocol definition should prevent
TIP working within any security environment.
7. TIP Protocol Transport Independence
It would be beneficial to some applications (though not
mandatory) to define the TIP protocol independently of the
transport for the TIP protocol itself. This would allow the TIP
protocol to flow over multiple different transport protocols.
The benefit is when using different transport protocols for the
application data, the same transport can be used for the TIP 2PC
protocol.
8. Recovery
Recovery semantics need to be defined sufficiently to avoid
ambiguous results in the event of any type of communications
transport failure.
References
[1] Internet Draft "The Transaction Internet Protocol"
J. Lyon et al.
[2] Transaction Processing: Concepts and Techniques
Morgan Kaufmann Publishers (ISBN 1-55860-190-2)
J. Gray, A. Reuter.
[3] X/Open CAE Specification, April 1995, Distributed Transaction
Processing: The TX Specification (ISBN 1-85912-094-6)
[4] X/Open Guide, November 1993, Distributed Transaction
Processing: Reference Model Version 2 (ISBN 1-85912-019-9)
[5] X/Open CAE Specification, December 1991, Distributed
Transaction Processing: The XA Specification
(ISBN 1-872630-24-3)
[6] Internet Draft "The SSL Protocol Version 3.0"
A. Freier, P. Karlton, P. Kocher.
Authors' Addresses
Jim Lyon Keith Evans
Microsoft Corporation Tandem Computers Inc, LOC 252-30
One Microsoft Way 5425 Stevens Creek Blvd
Redmond, WA 98052-6399, USA Santa Clara, CA 95051-7200, USA
Phone: +1 (206) 936 0867 Phone: +1 (408) 285 5314
Fax: +1 (206) 936 7329 Fax: +1 (408) 285 5245
Email: JimLyon@Microsoft.Com Email: Evans_Keith@Tandem.Com
Johannes Klein
Tandem Computers Inc.
10555 Ridgeview Court
Cupertino, CA 95014-0789, USA
Phone: +1 (408) 285 0453
Fax: +1 (408) 285 9818
Email: Klein_Johannes@Tandem.Com
Datatracker
draft-evans-tip-functions-00
Internet-Draft
