Individual Submission to ldup working group R. Harrison Internet Draft J. Sermersheim Document: draft-rharrison-lburp-02.txt Y. Dong Category: Standards Track Novell, Inc. June, 2000LDAP Bulk Update/Replication Protocol
Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.1. Abstract
The LDAP Bulk Update/Replication Protocol (LBURP) described in this document allows an LDAP client (a genuine client or an LDAP server acting as a client) to perform a bulk update to a replica on an LDAP server. The protocol groups a set of update operations using the LDAP framed protocol requests defined in [FRAMING] to notify the client that the update operations in the framed set are related. The update operations within the framed set are LDAPv3 extended operations each encapsulating a sequence number and one or more LDAPv3 update operations. The sequence number allows the server to process the update operations in the proper order even when they are sent asynchronously by the client, and the update operations can be grouped within the extended request to maximize the efficiency of client-server communication. The protocol may be used to initialize all of the entries in an LDAP replica or to incrementally update the existing entries in an LDAP replica. It is suitable for client utilities that need to efficiently initialize a replica with many entries or efficiently make a substantial set of update changes to a replica. It is also suitable as a protocol for replication between a single master replica and its slave replicas.2. Conventions used in this document
Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 1 LDAP Bulk Update/Replication Protocol June, 2000 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [ReqsKeywords]. The term supplier applies to an LDAP client or an LDAP server (acting as a client) that supplies a set of update operations to a consumer. The term consumer applies to an LDAP server that consumes (i.e. processes) the update operations sent to it by a supplier.3. Motivation for protocol
This protocol arose from the need to allow LDAP clients to efficiently present large quantities of updates to an LDAP server and have the LDAP server efficiently process them. LBURP introduces a minimum of new operational functionality to the LDAP protocol since the update requests sent by the client encapsulate standard LDAP [LDAPv3] update operations. However, this protocol greatly facilitates bulk updates by allowing the supplier to present the update operations asynchronously and still allow the consumer to maintain proper ordering of the operations. It also allows the consumer to recognize the supplier's intent to perform a potentially large set of update operations and then to change its processing strategy to be more efficient than it otherwise could be. In effect, this protocol gives a hint to the consumer that the LDAP operations framed within it can be treated in a special way because they are related to each other. The consumer may then take actions that would not otherwise be practical to speed the processing of the updates. Examples of such actions include refusing to perform operations for other clients during the update sequence and grouping update operations into a single transaction rather than applying them to the DIT singly. Additionally, this protocol deals with a common interoperability problem in this space caused by implementations having different requirements and abilities for handling linear and circular dependencies as entries are created. A common application of this protocol is anticipated to be the initialization or update of an LDAP replica from a set of records represented in an [LDIF] file. Due to the abilities of various implementations, these files often contain "out-of-order" records where an entry is created before its parent or it is made a member of a group before the member's entry has been created. Some implementations create temporary holding objects to deal with these issues, but others do not. This protocol allows the consumer to reorder update operations in a limited way to deal with such cases.4. Overview of protocol
Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 2 LDAP Bulk Update/Replication Protocol June, 2000 LBURP utilizes framing described in [FRAMING] to group a set of update operations to be applied to a partition. The update operations are sent via LDAPv3 extended operations, each containing a sequence number and a list of one or more update operations to be performed by the consumer. Except for the fact that they are grouped together as part of a larger LDAPv3 extended request, the update operations in each subset are normal LDAPv3 update operations and use the encoding specified in [LDAPv3].4.1. Update Initiation
The protocol is initiated when a supplier sends a StartFramedProtocolRequest extended operation to a consumer to notify it that the following LDAP update operations are to be treated as a stream of update information. The supplier includes the OID of the bulk update/replication protocol in the StartFramedProtocolRequest to tell the consumer that the stream of operations contained within the framing are LBURP operations. The consumer responds to the StartFramedProtocolRequest with a StartFramedProtocolResponse.4.2. Update Stream
After the consumer responds with a StartFramedProtocolResponse, the supplier sends a stream of LBURPOperation extended operations to the consumer. This stream MAY be sent asynchronously to maximize the efficiency of the transfer. The consumer responds to each LBURPOperation request with zero or more LBURPOperationStatus responses followed by an LBURPOperationDone response.4.2.1 LBURPOperation Request
Each LBURPOperation request contains a sequence number identifying its relative position within the update stream and an UpdateOperationList containing an ordered list of LDAPv3 update operations to be applied to the DIT. The sequence number enables the consumer to process LBURPOperation requests in the order they were sent by the supplier even when they are sent asynchronously. The consumer processes each LBURPOperation according to sequence number by applying the LDAPv3 update operations in its UpdateOperationList to the DIT in the order they are listed.4.2.2 LBURPOperationStatus Response
In cases where the consumer needs to deal with linear and circular dependencies, the consumer may temporarily defer the processing of some LDAPv3 update operations from the UpdateOperationList. In this case, the consumer processes the update operations that will not be deferred and sends an LBURPOperationStatus response containing the results of the updates that were processed along with a result code of deferred for the operations that have been deferred. Each time Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 3 LDAP Bulk Update/Replication Protocol June, 2000 that the consumer processes one or more update operations from the update operation list and there are remaining update operations, another LBURPOperationStatus response is sent.4.2.3 LBURPOperationDone Response
When all of the update operations from the UpdateOperationList have been applied to the DIT, the consumer sends an LBURPOperationDone response to the supplier indicating the success or failure of the update operations whose status have not already been reported to the supplier via LBURPOperationStatus responses. In most cases, the consumer will be able to process the entire UpdateOperationList immediately upon receipt. In this circumstance, no LBURPOperationStatus responses will be sent to the supplier and the LBURPOperationDone response will indicate the status for every operation in the UpdateOperationList.4.3. Update Termination
When the supplier has sent all of the LBURPOperation requests it needs to send, it sends an EndFramedProtocolRequest extended operation to the consumer to terminate the update stream. The consumer responds with an EndFramedProtocolResponse, and the update is complete.4.4. Update Styles
Two styles of update--full and incremental--are defined. Full update creates a completely new set of entries in the target partition. Any existing entries in that partition when the StartFramedProtocolRequest is received are removed before the LBURPOperation requests in the update stream are processed. The only update operation allowed in a full update stream is the add operation as defined in [LDAPv3]. Incremental update performs a series of incremental changes to the partition. Any existing entries in the partition are only affected if explicitly modified in some way by the update operations contained in the update stream. All of the update operations--add, modify, delete, and moddn--defined in [LDAPv3] are allowed in the incremental update stream.4.5. Applicability of Protocol
LBURP is designed to facilitate the bulk update of partitions on LDAPv3 servers. It can also be used to synchronize directory information between a single master replica and multiple slaves containing replicas of the same partition. Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 4 LDAP Bulk Update/Replication Protocol June, 2000 No attempt is made to deal with the issues associated with multiple- master replication environments (such as keeping modification times of attribute values) so that updates to the same entry on different replicas can be correctly ordered. For this reason, when LBURP is used for replication, proper convergence of the data between all replicas can only be assured in a single-master replication environment.5. Description of Protocol Flow
This section describes the LBURP protocol flow and the information contained in each protocol message. Throughout this section, the client or server acting as supplier is indicated by the letter "S", and the server acting as consumer is indicated by the letter "C". The construct "S -> C" indicates that the supplier is sending an LDAPv3 operation to the consumer, and "C -> S" indicates that the consumer is sending an LDAPv3 operation to the supplier. S -> C: LDAP bind operation (the identity and credentials used are implementation-defined) C -> S: Bind response S -> C: StartFramedProtocolRequest LDAPv3 extended operation. The parameters are: 1) OID of LBURP incremental or full update protocol as defined in this document. C -> S: StartFramedProtocolResponse LDAPv3 extended operation. The parameters are: 1) A suggested transactionSize (see section 6.2.1) S -> C: The supplier will send an update stream consisting of zero or more LBURPOperation LDAPv3 extended operations. The requests MAY be sent asynchronously. The parameters are: 1) A sequence number which specifies the order of this LBURPOperation request with respect to the other LBURPOperation requests in the update stream. 2) The UpdateOperationList, a list of one or more LDAPv3 update operations that are to be processed in the order they are listed. Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 5 LDAP Bulk Update/Replication Protocol June, 2000 The consumer processes the LBURPOperation requests in the order of their sequence numbers and applies the LDAPv3 update operations contained within each LBURPOperation to the DIT in the order they are listed. For the purposes of this discussion, when an LBURPOperation is received by the consumer, all of the update operations in its UpdateOperationList are considered to have a status of pending. When the consumer completes processing an operation from the UpdateOperationList (either succcessfully or unsuccessfully) the status of that operation changes from pending to completed. C -> S: The consumer sends zero or more intermediate LBURPOperationStatus messages to the supplier, one for each time it processes any of the pending update operations from the UpdateOperationList but also defers one or more of the pending update operations to deal with linear or circular dependencies. The result code for the LBURPOperationStatus message is always other (80). The LBURPOperationStatus message includes an LDAPResult element for each newly-completed update operation that did not complete successfully and for each pending update operation that is still being deferred from the original LBURPOperation. Update operations that are deferred remain in the pending state until they are processed. C -> S: When the consumer completes processing all pending update operations from the UpdateOperationList (whether some operations had been previously deferred or not), it sends an LBURPOperationDone response. If the consumer was able to successfully complete all pending update operations in the request, the consumer sends an LBURPOperationDone response to the supplier with a result code of success (0) indicating that the entire set of pending update operations for the LBURPOperation succeeded. If the consumer could not successfully decode any part of the LBURPOperation request, it sends a result code of protocolError (2) to the supplier. If one or more of the newly-completed update operations failed, the consumer sends an LBURPOperationDone message with a result code of other (80) and includes an Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 6 LDAP Bulk Update/Replication Protocol June, 2000 LDAPResult element for each of the failed update operations indicating the reason for failure. S -> C: After the supplier sends all of its LBURPOperation messages to the consumer, the supplier sends an EndFramedProtocolRequest message to the consumer. The parameters are: 1) A sequence number which is one greater than the sequence number of the last operation in the update stream. C -> S: The consumer responds by sending an EndFramedProtocolResponse message. S -> C: The supplier sends an unbind operation to the consumer to close the connection.6. Elements of Protocol
LBURP works within the framework described in [FRAMING] to initiate and terminate the protocol. An extended request, LBURPOperation, is used to send update operations from the supplier to the consumer. An extended partial response, LBURPOperationStatus, is used to keep the supplier apprised of the status of requests that have had one or more update operations temporarily deferred. An extended response, LBURPOperationDone is signals that an LBURPOperation is completed.6.1. StartFramedProtocolRequest Extended Operation
Section 4.1 of [FRAMING] defines the StartFramedProtocolRequest extended operation in terms of the [LDAPv3] ExtendedRequest as follows: ExtendedRequest ::= [APPLICATION 23] SEQUENCE { requestName [0] LDAPOID, requestValue [1] OCTET STRING OPTIONAL } The requestName portion of the StartFramedProtocolRequest must be the OID "2.16.840.1.113719.1.142.100.1". The requestValue of the StartFramedProtocolRequest must be set to the BER-encoding of the following: StartFramedProtocolRequestValue ::= SEQUENCE { framedProtocolOID LDAPOID, framedProtocolPayload OPTIONAL OCTET STRING } Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 7 LDAP Bulk Update/Replication Protocol June, 20006.1.1. framedProtocolOID
The framedProtocolOID is an OID that uniquely identifies the protocol framed by this operation. The framedProtocolOID for the LBURP Incremental Update Protocol is 2.16.840.1.113719.1.142.1.4.1. The framedProtocolOID for the LBURP Full Update Protocol is 2.16.840.1.113719.1.142.1.4.2.6.1.2. framedProtocolPayload
The framedProtocolPayload is an octet string that contains protocol- specific information. For LBURP, there is no framedProtocolPayload element for the StartFramedProtocolRequest extended operation.6.2. StartFramedProtocolResponse
Section 4.2 of [FRAMING] defines the StartFramedProtocolResponse in terms of the [LDAPv3] ExtendedResponse as follows: ExtendedResponse ::= [APPLICATION 24] SEQUENCE { COMPONENTS of LDAPResult, responseName [10] LDAPOID OPTIONAL, response [11] OCTET STRING OPTIONAL } The responseName of the StartFramedProtocolResponse must be the OID "2.16.840.1.113719.1.142.100.2". For LBURP, if the response element is included as part of a StartFramedProtocolResponse, it must be set to the BER-encoding of the following: LBURPStartFramedProtocolResponse ::= maxOperations maxOperations ::= INTEGER (0 .. maxInt) maxInt INTEGER ::= 2147483647 -- (2^^31 - 1) --6.2.1 maxOperations
The value of maxOperations is a hint sent by the consumer to tell the supplier the number of update operations per UpdateOperationList (see section 6.3.2) that it would like the supplier to send. The maxOperations element is optional. If the consumer does not send a maxOperations value, it MUST be prepared to accept any number of update operations per UpdateOperationList. A supplier MAY choose to ignore the maxOperations value sent by a consumer, however, if it Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 8 LDAP Bulk Update/Replication Protocol June, 2000 does, the supplier MUST NOT send more update operations in a single UpdateOperationList than the value in maxOperations.6.3. LBURPOperation
The LBURPOperation extended request is used to send a set of one or more LDAPv3 update operations from the supplier to the consumer along with sequencing information that enables the consumer to maintain the proper sequencing of multiple asynchronous LBURPOperation requests. An LDAPv3 Extended Request is defined in [LDAPv3] as follows: ExtendedRequest ::= [APPLICATION 23] SEQUENCE { requestName [0] LDAPOID, requestValue [1] OCTET STRING OPTIONAL } The responseName of the LBURPOperation must be the OID "2.16.840.1.113719.1.142.100.6". The requestValue of an LBURPOperation extended operation must be set the BER-encoding of the following: LBURPOperationValue ::= SEQUENCE { sequenceNumber INTEGER (1 .. maxInt), updateOperationList UpdateOperationList }6.3.1. sequenceNumber
The sequenceNumber is used to specify the ordering of LBURPOperation requests. This enables the consumer to know the order in which LBURPOperation requests must be processed even if it receives them in a sequence different from that in which they were sent from the supplier. The supplier MUST set the value of sequenceNumber of the first LBURPOperation to 1, and MUST increment the value of sequenceNumber for each succeeding LBURPOperation.6.3.2. UpdateOperationList
The UpdateOperationList is a list of one or more standard LDAP update requests and is defined as follows: UpdateOperationList ::= SEQUENCE { updateOperation CHOICE { addRequest AddRequest, modifyRequest ModifyRequest, delRequest DelRequest, modDNRequest ModifyDNRequest, } Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 9 LDAP Bulk Update/Replication Protocol June, 2000 controls [0] Controls OPTIONAL } AddRequest, ModifyRequest, DelRequest, and ModifyDNRequest are standard LDAP update requests as defined in sections 4.6, 4.7, 4.8, and 4.9 of [LDAPv3]. For the LBURP Full Update Protocol, the only valid choice is addRequest. Clients MUST not include any other choice in LBURPOperations sent as part of an LBURP Full Update Protocol update stream. For the LBURP Incremental Update Protocol, all of the choices listed in UpdateOperationList are valid and can be freely intermixed within an LBURP Incremental Update Protocol update stream The LDAP update requests in the UpdateOperationList MUST be applied to the DIT in the order in which they are listed (subject to the exceptions provided for in section 7.1 to deal with linear and circular dependency issues).6.4. LBURPOperationStatus
An LBURPOperationStatus message is sent from the consumer to the supplier to give the results for all newly-completed update operations from the UpdateOperationList of an LBURPOperation and to tell the supplier about one or more update operations from the UpdateOperationList that have been deferred. An LBURPOperationStatus message is sent each time the consumer processes one or more of the pending update operations and also defers the processing of one or more of the update operations from the UpdateOperationList. The LBURPOperationStatus message gives the results for the newly-completed update operations and lists the remaining update operations still in the pending state because they have been deferred. This allows the supplier to know which update operations have been completed (either successfully or unsuccessfully) at any given point in time and also keep track of the update operations that have been deferred. An LDAPv3 Extended Partial Response is defined in [PartialResponse] as follows: ExtendedResponse ::= [APPLICATION 24] SEQUENCE { COMPONENTS of LDAPResult, responseName [10] LDAPOID OPTIONAL, response [11] OCTET STRING OPTIONAL } Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 10 LDAP Bulk Update/Replication Protocol June, 2000 The responseName of the LBURPOperationStatus message must be the OID "2.16.840.1.113719.1.142.100.9". The resultCode value for an LBURPOperationStatus message MUST be other (80) to signal that an OperationResults element is contained in the message.6.4.1. OperationResults
A response element is always included in an LBURPOperationStatus message and must be set to the BER-encoding of the following: OperationResults ::= SEQUENCE of OperationResult OperationResult ::= SEQUENCE { operationNumber INTEGER, ldapResult LDAPResult } An OperationResult element is included for (a) each operation from the UpdateOperationList that has been deferred to deal with linear or circular dependency issues, and (b) each newly-completed operation from the UpdateOperationList that failed during processing. Update operations for which an LBBURPOperationStatus have already been sent are not included again, and any newly-completed update operation which is not listed as deferred or failed MUST be assumed to have succeeded.6.4.1.1. operationNumber
The operationNumber identifies the LDAPv3 update operation from the UpdateOperationList of the LBURPOperation that was deferred or that failed. Operations are numbered beginning at 1.6.4.1.2. ldapResult
The ldapResult included in the OperationResult is the same ldapResult that would be sent for the update operation that failed if it had failed while being processed as a normal LDAPv3 update operation.6.5. LBURPOperationDone
Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 11 LDAP Bulk Update/Replication Protocol June, 2000 An LBURPOperationDone message is sent from the consumer to the supplier to signal that all of the update operations from the UpdateOperationList of an LBURPOperation have been completed and to give the results for all newly-completed update operations from that list. In the simplest case, all of the update operations from the UpdateOperationList are processed as a set by the consumer and an LBURPOperationDone message is sent back to the supplier indicating the status for the entire list. In this case, no LBURPOperationStatus message is needed because none of the operations are deferred. In cases where one or more operations were previously deferred, the LBURPOperationDone message indicates that any remaining deferred operations from the UpdateOperationList have been completed, and the LBURPOperationDone message is sent back to the supplier indicating the status for those deferred operations that have now been completed. An LDAPv3 extended response is defined in [LDAPv3] as follows: ExtendedResponse ::= [APPLICATION 24] SEQUENCE { COMPONENTS of LDAPResult, responseName [10] LDAPOID OPTIONAL, response [11] OCTET STRING OPTIONAL } The responseName of the LBURPOperationDone MUST be the OID "2.16.840.1.113719.1.142.100.7". If the consumer server cannot successfully decode the LBURPOperation in its entirety, the resultCode for the LBURPOperationDone is set to protocolError (2) and response element is omitted. No updates from the LBURPOperation should be committed to the DIT in this circumstance. If the status of all of the update operations being reported by an LBURPOperationDone message is success (0), the resultCode of the LBURPOperationDone message is set to success (0) and the response element is omitted. If the status of of any of the update operations being reported by an LBURPOperationDone message is something other than success (0), the resultCode for the entire LBURPOperationDone is set to other (80) to signal that an OperationResults element is in included in the message.6.5.1 OperationResults
Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 12 LDAP Bulk Update/Replication Protocol June, 2000 The OperationResults element included in an LBURPOperationDone message is precisely the same as the OperationResults element used for the LBURPOperationStatus message except that no deferred update operations can be listed because the LBURPOperationDone message is only sent when all of the update operations in the UpdateOperationList have completed. See section 6.4.1. for details on the OperationResults element. Update operations for which an LBBURPOperationStatus have already been sent are not included again, and any newly-completed update operation which is not listed as failed MUST be assumed to have succeeded.6.6. EndFramedProtocolRequest
Section 4.3 of [FRAMING] defines the EndFramedProtocolRequest extended operation in terms of the [LDAPv3] ExtendedRequest as follows: ExtendedRequest ::= [APPLICATION 23] SEQUENCE { requestName [0] LDAPOID, requestValue [1] OCTET STRING OPTIONAL } The requestName of the EndFramedProtocolRequest must be the OID "2.16.840.1.113719.1.142.100.4". For LBURP, the requestValue of the EndFramedProtocolRequest must be set to the BER-encoding of the following: LBURPEndFramedProtocolRequestValue::= SEQUENCE { sequenceNumber INTEGER (1 .. maxInt) }6.6.1. sequenceNumber
The value in sequenceNumber is one greater than the last LBURPOperation sequence number in the update stream. It allows the server to know when it has received all outstanding asynchronous LBURPOperations.6.7. EndFramedProtocolResponse
Section 4.4 of [FRAMING] defines the EndFramedProtocolResponse in terms of the [LDAPv3] ExtendedResponse as follows: ExtendedResponse ::= [APPLICATION 24] SEQUENCE { COMPONENTS of LDAPResult, responseName [10] LDAPOID OPTIONAL, response [11] OCTET STRING OPTIONAL } Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 13 LDAP Bulk Update/Replication Protocol June, 2000 The responseName of the EndFramedProtocolResponse must be the OID "2.16.840.1.113719.1.142.100.5". For LBURP, there is no response element for the EndFramedProtocolResponse extended operation.7. Semantics of Full and Incremental Update Protocols
Most of the semantics of LBURP are common to both the full and incremental update protocols although there are some semantic differences between the two.7.1. Semantics Common to Both Full and Incremental Update Protocols
The LBURPOperation requests in the update stream, plus the initial state of entries in the consumer replica in the case of incremental update, collectively represent the desired final state of the consumer replica. The consumer server may take any action required to efficiently achieve this desired final state of the replica. Examples of such actions include reordering LDAP requests to ensure that linear dependencies within the update stream are correctly ordered, breaking an LDAP modification request into two or more separate modification requests to adequately resolve a circular dependency within the update stream, and aggregating LDAP update operations to process them more efficiently. When taking actions of this sort, the server MUST NOT reorder LDAP requests in such a way that the desired final state is different from that which would have been achieved before the action was taken. The LBURPOperation requests that form the update stream MAY be sent asynchronously by the supplier to the consumer. This means that the supplier need not wait for an LBURPOperationDone response for one LBURPOperation request before sending the next LBURPOperation request. Suppliers MUST NOT send request other than an LBURPOperation between the StartFramedProtocolRequest and EndFramedProtocolRequest. If a consumer received any other LDAP request between the StartFramedProtocolRequest and the EndFramedProtocolRequest, it MUST send a response of protocolError (2) for the offending request. When the LBURP update stream contains a request that affects multiple DSAs, the consumer MAY choose to perform the request or return a resultCode value of affectsMultipleDSAs (71). As with any LDAPv3 operation, a consumer MAY send a resultCode value of referral (10) as part of the OperationResult element for any operation on an entry that it does not contain. If the consumer is configured to do so, it MAY chain on behalf of the supplier to complete the update operation instead. Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 14 LDAP Bulk Update/Replication Protocol June, 2000 While a consumer server is processing an LBURP update stream, it MAY choose to not service LDAP requests on other connections for the replica involved in the LBURP operation. This provision is designed to allow implementers the freedom to implement highly-efficient methods of handling the update stream without being constrained by the need to maintain a live, working DIT database while doing so. If the update sequence fails for some reason (e.g. a connection is abruptly broken in the middle of an otherwise-successful update sequence) the consumer MUST leave the entries that have been completed in the DIT, thereby leaving the replica in a known state. Since both the full and incremental update protocols use sequences of standard LDAPv3 operations to transmit update information, no attempt is made by the protocol to include the information needed to support full multi-master replication as defined by [LDUP RUP]. Unless attribute values specifying timestamps are included as part of the update stream, updates made using LBURP are treated the same as other LDAP operations wherein they are deemed to occur at the present. Consumers MAY store timestamp values sent by suppliers.7.2. Semantics of the Full Update Protocol
Full update creates a completely new set of entries in that replica; any existing entries in that replica at the start of the LBURP session are removed before the LDAP requests in the update stream are processed. The only LDAP operation allowed in a full update stream is add. Clients MUST NOT send any other type of update operation in the full update stream. If any other LDAP update operation is included in an UpdateOperationList, the server MUST return protocolError (2) as the response for the operation.7.3. Semantics of the Incremental Update Protocol
Incremental update performs a series of incremental changes to the replica; existing entries in the replica are only affected if explicitly modified in some way by the update operations contained in the update stream. All LDAPv3 update operations--add, modify, delete, and moddn--are allowed in the incremental update stream.8. Notes To Implementers
If a consumer chooses to take a replica off-line during an LBURP update, it is RECOMMENDED that the consumer refer non-LBURP requests Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 15 LDAP Bulk Update/Replication Protocol June, 2000 for the off-line replica to another server containing a replica of the partition. Implementations MAY choose to perform the operations in the update stream with special permissions to improve performance. Consumer implementations SHOULD include functionality to detect and terminate connections on which an LBURP session has been initiated but information (such as an LBURPOperation or the LDAPEndFramedProtocolRequest) needed to complete the LBURP session is never received. A timeout is one mechanism that can be used to accomplish this. Consumer implementations SHOULD NOT unnecessarily defer the processing of update operations. Guidance on the valid reasons for deferring update operations is provided in section 7.1.9. Security Considerations
Implementations should ensure that a supplier making an LBURP request is bound with appropriate permissions. There is a potential for loss of data, especially with the full update protocol, which removes all entries in a replica as part of its operation. Second, forcing the removal of all entries in a replica may consume large amounts of server resources. Third, unlike other replication protocols, no existing replication agreement between supplier and consumer is required. These risks increase if the consumer server also processes the update stream with special permissions to improve performance. For these reasons, implementers should carefully consider which identities or permissions are required to perform LBURP operations and take steps to ensure that only connections bound with appropriate permissions are allowed to perform them. The data contained in the update stream may contain passwords and other sensitive data. Care should be taken to properly safeguard this information while in transit between supplier and consumer. As with any asynchronous LDAP operation, it may be possible for an LBURP supplier to send asynchronous LBURPOperation requests to the consumer faster than the consumer can process them. Consumer implementers should take steps to prevent LBURP suppliers from being able interfere with the normal operation of a consumer server by issuing a rapid stream of asynchronous LBURPOperation requests.10. References
[FRAMING] Stokes, Ellen, and Gordon Good, _Extended Operations for Framing LDAP Operations_, Unpublished Internet-Draft: ldup- framing.txt. Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 16 LDAP Bulk Update/Replication Protocol June, 2000 [LDAPv3] Wahl, M., Howes, T., and S. Kille, "Lightweight Directory Access Protocol (v3)", RFC 2251, December 1997. [LDIF] Gordon Good, "LDAP Data Interchange Format (LDIF) - Technical Specification", RFC 2849. [PartialResponse] Roger Harrison. "Extended Partial Response Protocol Enhancement to LDAP v3", Internet-Draft: draft-rharrison-ldap-extPartResp- 01.txt , June, 2000. [ReqsKeywords] Scott Bradner. "Key Words for use in RFCs to Indicate Requirement Levels", RFC 2119.11. Author's Addresses
Roger Harrison Novell, Inc. 1800 S. Novell Place Provo, UT 84606 +1 801 861 2642 roger_harrison@novell.com Jim Sermersheim Novell, Inc. 1800 S. Novell Place Provo, UT 84606 +1 801 861 3088 jimse@novell.com Yulin Dong Novell, Inc. 1800 S. Novell Place Provo, UT 84606 +1 801 861 4940 ydong@novell.com Appendix A - Document Revision HistoryA.1. draft-rharrison-lburp-00.txt
Initial DraftA.2. draft-rharrison-lburp-01.txt
Adjusted LBURP protocol to use extended requests for all operations. LDAP update operations are now encapsulated within the LBURPOperation request for two reasons: (1) To allow the inclusion Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 17 LDAP Bulk Update/Replication Protocol June, 2000 of operation ordering information. This allows LDAP servers to maintain the proper ordering of updates even in cases where multi- threaded stacks present update operations to the server out-of- sequence. (2) To allow multiple update operations to be sent from client to server in a single request. This was a natural evolution of the changes made for (1) and allows the protocol to make more efficient use of network bandwidth, Converted references to LDUP extended operations to use a new LDAP Framed Operations Protocol. Specified OIDs used for the protocol and extended operations. Changed requirement that a server "MUST NOT" service non-LBURP requests during a full update to a "MAY choose to not" service non- LBURP requests during a full update. This gives implementers the option to do what is needed without imposing a requirement that may not be needed by some implementations.A.3. draft-rharrison-lburp-02.txt
Clarified error responses in cases where one or more of the update operations in the UpdateOperationList of the LBURPOperation fail. Utilized the extended partial response and the LBURPOperationStatus message to allow the consumer to give status on deferred operations and documented this in the protocol flow and elements of protocol. Updated the ASN.1 definition of UpdateOperationList to allow the inclusion of a control on each individual update operation. Made cosmetic changes to the names of the protocol elements to clarify their meanings. Clarified the semantics of the protocol and added additional notes to implementers and security considerations based on implementation and field experience. Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implmentation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 18 LDAP Bulk Update/Replication Protocol June, 2000 copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Individual Submission - Expires December 22, 2000 Harrison, Sermersheim & Dong 19
Datatracker
draft-rharrison-lburp-02
This is an older version of an Internet-Draft that was ultimately published as RFC 4373.
