Internet Draft R. Megginson, Editor Document: <draft-ietf-ldup-lcup-02.txt> M. Smith Category: Proposed Standard Netscape Communications Corp. O. Natkovich Yahoo J. Parham Microsoft Corporation November 2001 LDAP Client Update Protocol Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract This document defines the LDAP Client Update Protocol (LCUP). The protocol is intended to allow an LDAP client to synchronize with the content of a directory information tree (DIT) stored by an LDAP server and to be notified about the changes to that content. 2. Conventions used in this document In the protocol flow definition, the notation C->S and S->C specifies the direction of the data flow from the client to the server and from the server to the client respectively. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [KEYWORDS]. 3. Overview The LCUP protocol is intended to allow LDAP clients to synchronize with the content stored by LDAP servers. The problem areas addressed by the protocol include: - mobile clients that maintain a local read-only copy of the directory data. While off-line, the client uses the local copy of the data. When the client connects to the network, it synchronizes with the current directory content and can be optionally notified about the changes that occur while it is on- line. For example, a mail client can maintain a local copy of the corporate address book that it synchronizes with the master copy whenever the client gets connected to the corporate network. - applications intending to synchronize heterogeneous data stores. A meta directory application, for instance, would periodically retrieve a list of modified entries from the directory, construct the changes and apply them to a foreign data store. - clients that need to take certain actions when a directory entry is modified. For instance, an electronic mail repository may want to perform a "create mailbox" task when a new person entry is added to an LDAP directory and a "delete mailbox" task when a person entry is removed. The problem areas not being considered: - directory server to directory server synchronization. The replication protocol that is being defined by the LDUP IETF working group should be used for this purpose. Several features of the protocol distinguish it from LDUP replication. First, the server does not maintain any state information on behalf of its clients. The clients are responsible for storing the information about how up to date they are with respect to the server's content. Second, no predefined agreements exist between the clients and the servers. The client decides when and from where to retrieve the changes. Finally, the server never pushes the data to the client; the client always initiates the update session during which it pulls the changes from the server. The set of clients that are allowed to synchronize with an LDAP server is determined by the server defined policy. There are currently several protocols in use for LDAP client server synchronization. While each protocol addresses the needs of a particular group of clients (e.g., on-line clients or off-line clients) none satisfies the requirements of all clients in the target group. For instance, a mobile client that was off-line and wants to become up to date with the server and stay up to date while connected can't be easily supported by any of the existing protocols. Megginson, et. al. Proposed Standard - Expires: May 2002 2 A server can define a naming context or some part thereof to participate in LCUP. This document will refer to this as an LCUP Context. For example, LDUP defines a replica, a part of the DIT which may participate in replication. The LCUP context may be coincident with the replicated area, depending on the server's implementation. It is assumed that most server implementations of LCUP will make use of the server's underlying replication mechanism, but this does not have to be LDUP compliant. 4. Protocol Specification This section describes the protocol elements and the protocol flow. 4.1 Unique Identifiers Distinguished names can change, so are therefore unreliable as identifiers. The server SHOULD assign a Unique Identifier to each entry as it is created. This identifier will be stored as an operational attribute of the entry, named `entryUUID'. The entryUUID attribute is single valued. If the client wants to use entryUUID, it should supply entryUUID in the list of attributes to return in the LCUP search (described below). A consistent algorithm for generating such unique identifiers may be standardized at some point in the future. The definition of the entryUUID attribute type, written using the BNF form of AttributeDescription described in RFC 2252 [RFC2252] is: ( OID-To-Be-Specified NAME `entryUUID' DESC `unique entry identifier' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE NO-USER-MODIFICATION USAGE directoryOperation ) 4.2 LCUP Cookie Value The LCUP protocol uses a cookie to hold the state of the client's data with respect to the server's data. The LCUP Cookie is a value of the following ASN.1 type: LCUPCookie ::= SEQUENCE { scheme LDAPOID, value OCTET STRING OPTIONAL } scheme - this is the OID which identifies the format of the value. The scheme OID, like all object identifiers, MUST be unique for a given cookie scheme. The cookie value may be opaque or it may be exposed to LCUP clients. For cookie schemes that expose their value, the preferred form of documentation is an RFC. It is expected that there will be one or more standards track cookie schemes where the value format is exposed and described in detail. Megginson, et. al. Proposed Standard - Expires: May 2002 3 value - this is the actual data describing the state of the client's data. This value may be opaque, or its value may have some well known format, depending on the scheme. The cookie value MUST be included except when a client has no stored state; i.e., when the client is requesting a full synchronization. When the server sends back a cookie, the cookie value MUST be present. Further uses of the LCUP Cookie value are described below. 4.3 LCUP Cookie Schemes Operational Attribute The OIDs of the supported LCUP cookie schemes SHOULD be published using the following operational attribute: ( OID-TBD NAME 'lcupCookieScheme' EQUALITY objectIdentifierMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.38 NO-USER-MODIFICATION USAGE directoryOperation ) The lcupCookieScheme operational attribute MUST be present in the root DSE. The lcupCookieScheme operational attribute MAY be present in every directory entry that may be used as the baseObject for a search request that contains an LCUP clientUpdate control. If a client wants to determine what LCUP cookie schemes are supported, it MAY use a base object search to read the lcupCookieScheme attribute from the entry that will be used as the baseObject in subsequent LCUP sessions, then query the root DSE if the lcupCookieScheme was not found in the base entry. Clients SHOULD NOT search for entries that contain lcupCookieScheme values; rather, it is RECOMMENDED that servers support LCUP sessions based at as many different entries as possible. Each value of the attribute will be a list of OIDs of the cookie schemes followed by the DN of the LCUP context which supports the schemes. The delimiter will be a single space character. For example: lcupCookieScheme: 1.2.3.4 5.6.7.8 dc=mycorp, dc=com Everything after the last space after the last OID will be the LCUP Context DN. If the attribute is present in a regular directory entry in an LCUP Context, the values corresponding to DNs other than the LCUP Context containing the entry MAY be omitted. 4.4 Client Update Control Value A client initiates a synchronization session with a server by attaching a clientUpdate control to a search operation. The search specification determines the part of the directory information tree (DIT) the client wishes to synchronize with, the set of attributes it is interested in and the amount of data the client is willing to receive. The clientUpdate control contains the client's synchronization specification. The controlType field for the Megginson, et. al. Proposed Standard - Expires: May 2002 4 clientUpdate control is ClientUpdateControlOID (to be assigned). The controlValue is an OCTET STRING, whose contents are the bytes of the BER encoding of the following: ClientUpdateControlValue ::= SEQUENCE { updateType ENUMERATED { synchronizeOnly (0), synchronizeAndPersist (1), persistOnly (2) }, cookie LCUPCookie OPTIONAL } updateType - specifies the type of update requested by the client synchronizeOnly - the server sends all the data needed to synchronize the client with the server, then closes the connection synchronizeAndPersist - the server sends all the data needed to synchronize the client with the server, then leaves open the connection, sending to the client any new added, modified, or deleted entries which satisfy the search criteria. persistOnly - the server does not synchronize the data with the client but leaves open the connection and sends over any new added, modified, or deleted entries which satisfy the search criteria. cookie - a value that represents the current state of the client's data. If a cookie is provided, the server MUST use the enclosed scheme throughout the duration of the LCUP session or until an LCUP context boundary is crossed, since a new cookie may be required in that case. If the value or scheme part of the cookie is invalid, the server MUST return immediately with a SearchResultDone message with the clientUpdateDone control attached with the reason set to the value of lcupInvalidCookie (see below). Also, the LDAP result code MUST be unwillingToPerform. If the scheme part of the cookie is a valid OID, but is not supported, the server MUST return immediately with a SearchResultDone message with the clientUpdateDone control attached with the reason set to the value of lcupUnsupportedScheme (see below). Also, the LDAP result code MUST be unwillingToPerform. If the cookie is omitted, the server MAY use any scheme it supports. 4.5 Entry Update Control Value In response to the client's synchronization request, the server returns one or more SearchResultEntry PDU that fits the client's specification. Each SearchResultEntry PDU also contains an entryUpdateControl which describes the LCUP state of the returned entry. To represent a deleted entry, the server attaches an Megginson, et. al. Proposed Standard - Expires: May 2002 5 entryUpdate control to the corresponding SearchResultEntry. The SearchResultEntry corresponding to a deleted entry MUST contain a valid DN and MAY contain a valid Unique Identifier but, to reduce the amount of data sent to the client, it SHOULD not contain any other attributes. Distinguished names can change, so are therefore unreliable as identifiers. A Unique Identifier MAY therefore be assigned to each entry as it is created. The Unique Identifier allows the client to uniquely identify entries even in the presence of modifyDN operations. The Unique Identifier is carried in the entryUUID attribute. For returned SearchResultEntry PDUs other than deleted entries, the client MAY request that the Unique Identifier attribute be returned by specifying it in the attribute list to be returned by the search request. If the Unique Identifier is not returned, the client MAY use the entry DN to keep track of returned entries. Furthermore, the server may elect to periodically return to the client the cookie that represents the state of the client's data. This information is useful in case the client crashes or gets disconnected. The cookie SHOULD be present in every entryUpdate control sent to the client to insure ease of synchronization. The cookie is also provided in the entryUpdate control. The controlType field for the entryUpdate control is EntryUpdateControlOID (to be assigned). The controlValue is an OCTET STRING, whose contents are the bytes of the BER encoding of the following: EntryUpdateControlValue ::= SEQUENCE { stateUpdate BOOLEAN, entryDeleted BOOLEAN, cookie LCUPCookie OPTIONAL } stateUpdate - if set to TRUE, indicates that the entry to which the control is attached contains no changes and it is sent only to communicate to the client the new cookie. In this case, the entryDeleted field MUST be ignored and the cookie field MUST contain the updated cookie. This feature allows updating the client's cookie when there are no changes that effect the client's data store. Note that the control MUST be attached to a valid SearchResultEntry, i.e. the entry should contain a valid dn. The server MAY send the entry named by the baseObject from the client's search request. entryDeleted - if set to TRUE, indicates that the entry to which the control is attached was deleted. The server MAY also set this to TRUE if the entry has left the client's search result set. As far as the client is concerned, a deleted entry is no different than an entry which has left the result set. cookie - the LCUP cookie value that represents the current state of the client's data. 4.6 Client Update Done Control Value Megginson, et. al. Proposed Standard - Expires: May 2002 6 When the server has finished processing the client's request, it attaches a clientUpdateDone control to the SearchResultDone message and sends it to the client. However, if the SearchResultDone message contains a resultCode which is not success, the clientUpdateDone control MAY be omitted. The controlType field for the clientUpdateDone control is ClientUpdateDoneControlOID (to be assigned). The controlValue is an OCTET STRING, whose contents are the bytes of the BER encoding of the following: ClientUpdateDoneControlValue ::= SEQUENCE { reason INTEGER, reasonText LDAPString, cookie LCUPCookie OPTIONAL } reason - reason for terminating the operation. The following values are defined: lcupSuccess (0) the operation was successfully processed lcupResourcesExhausted (1) the server is running out of resource lcupSecurityViolation (2) the client is suspected of malicious actions lcupInvalidCookie (3) invalid cookie was supplied by the client - both/either the scheme and/or the value part was invalid lcupUnsupportedScheme (4) The scheme part of the cookie is a valid OID but is not supported by this server lcupClientDisconnect (5) client requested search termination using the stopClientUpdate request (defined below) lcupReloadRequired (6) indicates that client data needs to be reinitialized. This reason is returned if the server does not contain sufficient information to synchronize the client or that the server's data was reloaded since the last synchronization session reasonText - The reasonText field of this construct may, at the server's option, be used to return a string containing a textual, human-readable (terminal control and page formatting characters should be avoided) error diagnostic. As this error diagnostic is not standardized, implementations MUST NOT rely on the values returned. If the server chooses not to return a textual diagnostic, the reasonText field of the ClientUpdateDoneControlValue MUST contain a zero length string. The reasonText should be limited to characters in the range 0x00 to 0x7F. cookie - the LCUP cookie value that represents the current state of the client's data. Although this value is OPTIONAL, it MUST be set Megginson, et. al. Proposed Standard - Expires: May 2002 7 in the ClientUpdateDoneControlValue if the reason is lcupSuccess or lcupClientDisconnect and the LDAP search result code is success. This provides a good "checksum" of what the server thinks the state of the client is. If some error occurred, either an LDAP search error (e.g. insufficientAccessRights) or an LCUP error (e.g. lcupUnsupportedScheme), the cookie MAY be omitted. If server resources become tight, the server can terminate one or more search operations by sending a SearchResultDone message to the client(s). Unless the client sets the updateType field to persistOnly, the server attaches a clientUpdateDone control that contains the cookie that corresponds to the current state of the client's data and the value of the reason field is set to lcupResourcesExhausted. A server set policy is used to decide which searches to terminate. This can also be used as a security mechanism to disconnect clients that are suspected of malicious actions, but if the server can infer that the client is malicious, the server should return lcupSecurityViolation in the reason field of the response. 4.7 Stop Client Update Request and Response The Stop Client Update operation is an LDAPv3 Extended Operation [RFC2251, Section 4.12] and is identified by the OBJECT IDENTIFIER stopClientUpdateRequestOID (to be assigned). This section details the syntax of the protocol. An LDAPv3 Extended Request is defined in [LDAPv3] as follows: ExtendedRequest ::= [APPLICATION 23] SEQUENCE { requestName [0] LDAPOID, requestValue [1] OCTET STRING OPTIONAL } If the client needs to terminate the synchronization process and it wishes to obtain the cookie that represents the current state of its data, it issues a stopClientUpdateRequest extended operation. The operation carries the following data. The extended operation requestValue is an OCTET STRING, whose contents are the bytes of the BER encoding of the following: StopClientUpdateRequestValue ::= MessageID StopClientUpdateRequestValue - the message ID of the search that included the original clientUpdate control The server responds immediately with a stopClientUpdateResponse extended operation that carries no data, and an OBJECT IDENTIFIER of stopClientUpdateResponseOID (to be assigned). The server MAY send any pending SearchResultEntry PDUs if the server cannot easily abort or remove those search results from its outgoing queue. The server SHOULD send as few of these remaining SearchResultEntry PDUs as possible. Finally, the server sends the message SearchResultDone with the clientUpdateDone control attached. The value of the reason Megginson, et. al. Proposed Standard - Expires: May 2002 8 in the clientUpdateDone control value MUST be either an error code (some value other than lcupSuccess) or lcupClientDisconnect. The stopClientUpdateResponse is sent only to satisfy LDAP requirement that every server must issue an extended response for each extended request it receives. If the client is not interested in the state information, it can simply abandon the search operation or disconnect from the server. The requestName portion of the stopClientUpdate must be the OID stopClientUpdateOID (to be assigned). The requestValue is the message ID corresponding to the client's search request. If the message ID is not valid, the server MUST send back to the client an LDAP error code of unwillingToPerform. 4.8 Protocol Flow The client server interaction can proceed in three different ways depending on the client's requirements. Protocol flows beginning with an asterisk (*) are optional or conditional. If the client's intent is not to synchronize data but to trigger actions in response to directory modifications, the protocol proceeds as follows: C->S Sends a search operation with a clientUpdate control attached. The search specification determines the part of the DIT the client wishes to synchronize with and the set of attributes it is interested in. The updateType field of the control value should be set to persistOnly. *S->C If there is an error (invalid search scope, invalid cookie) the server returns the appropriate error codes and terminates the request (SearchResultDone message with optional clientUpdateDone control) S->C Sends change notification to the client for each change to the data within the client's search specification. Each SearchResultEntry may have an entryUpdate control attached. *S->C If the server starts to run out of resources or the client is suspected of malicious actions, the server SHOULD terminate the search operation by sending to the client a SearchResultDone message with clientUpdateDone control attached. The control contains the reason field set to lcupResourcesExhausted or lcupSecurityViolation depending on the reason for termination. The server MAY provide more details to the client via the reasonText field of the control. *C->S If the client receives lcupResourcesExhausted error from the server, it MUST wait for a while before attempting another synchronization session with the server. It is RECOMMENDED that clients use an exponential backoff strategy. C->S The client terminates the search. The client can do this by abandoning the search operation, disconnecting from the server, or by sending the stopClientUpdate extended operation. *S->C If the server receives the stopClientUpdate extended op, it will immediately send back the stopClientUpdate extended op Megginson, et. al. Proposed Standard - Expires: May 2002 9 response *S->C If the client sent the stopClientUpdate extended op, the server MAY send any pending SearchResultEntry PDUs in its outgoing queue *S->C If the client sent the stopClientUpdate extended op, after the server sends the response and any pending SearchResultEntry PDUs, the server sends the SearchResultDone message with the clientUpdateDone control attached. The value of the reason field of the clientUpdateDone control value will be either lcupClientDisconnect or some lcup error code (not lcupSuccess). S->C Stops sending changes to the client and closes the connection. If the client's intent is to synchronize with the server and then disconnect, the protocol proceeds as follows: C->S Sends a search operation with the clientUpdate control attached. The search specification determines the part of the DIT the client wishes to synchronize with, the set of attributes it is interested in and the amount of data the client is willing to receive. If this is the initial synchronization session, the client either does not provide a cookie or provides a cookie with no value; otherwise, the cookie field of the control is set to the cookie received from the server at the end of the last synchronization session. If the scheme field of the cookie was provided, the server MUST use that scheme throughout the duration of the LCUP session or until an LCUP boundary is crossed, since the server will usually require a different cookie in that case anyway. (Note that the client can synchronize with different servers during different synchronization sessions.) The updateType field of the control value is set to synchronizeOnly. *S->C If there is an error (invalid search scope, invalid cookie) the server returns the appropriate error codes and terminates the request (SearchResultDone message with optional clientUpdateDone control) *S->C If no cookie is specified in the clientUpdate control, or if the value field of the cookie is empty, the server sends all data that matches the client's search specification followed by the SearchResultDone message with a clientUpdateDone control attached. The control contains the cookie that corresponds to the current state of the client's data and the reason flag set to lcupSuccess. *S->C If an invalid cookie is specified, the server sends the SearchResultDone message with clientUpdateDone control attached. The reason field of the control is set to lcupInvalidCookie and the reasonText field MAY contain explanation of the error. *S->C If a valid cookie is specified and the data that matches the search specification has been reloaded or the server does not contain enough state information to synchronize the client, the server sends a SearchResultDone message with clientUpdateDone control attached. The reason field of the control is set to lcupReloadRequired and the reasonText field Megginson, et. al. Proposed Standard - Expires: May 2002 10 MAY contain explanation of the error. *S->C If the cookie is valid and the client is up to date, the server sends a success response to the client. S->C If the cookie is valid and there is data to be sent, the server sends the modified entries to the client. Each SearchResultEntry contains the attributes requested by the client in the search specification regardless of whether they were modified. An entryUpdate control with the entryDeleted field set to TRUE MUST be attached to every deleted entry. The server may also periodically attach an entryUpdate control to the entries sent to the client to indicate the current state of the client's data. In that case, the cookie field of the control represents the state of the client's data including the entry to which the control is attached. Once all the changes are sent, the server sends a SearchResultDone with the clientUpdateDone control attached. The control contains the cookie that represents the current state of the client's data. The reason field of the control is set to lcupSuccess. The client stores the cookie received from the server until the next synchronization session. *C->S If the reason field of the control is set lcupReloadRequired, the client clears its data store and repeats the synchronization process by sending the search operation with clientUpdate control that contains no cookie, or that contains a cookie with no value field. If the client's intent is to be synchronized with the server and stay notified about data modifications, the protocol proceeds as follows: C->S The client behaves exactly as in the previous case except it sets the updateType field in the control value to synchronizeAndPersist. S->C The server behaves exactly as in the previous case except the connection is kept open after the initial set of changes is sent to the client. A SearchResultDone message is not sent to the client; instead, the server keeps sending changes to the client. *S->C If the server starts to run out of resources or the client is suspected of malicious actions, the server SHOULD terminate the search operation by sending to the client a SearchResultDone message with clientUpdateDone control attached. The control contains the reason field set to lcupResourcesExhausted or lcupSecurityViolation depending on the reason for termination. The server MAY provide more details to the client via the reasonText field of the control. *C->S If the client receives lcupResourcesExhausted error from the server, it MUST wait for a while before attempting another synchronization session with the server. We recommend exponential backoff strategy. C->S Sends a stopClientUpdateRequest extended operation to the server to terminate the synchronization session. S->C Responds with a stopClientUpdateResponse extended operation followed by a SearchResultDone with the clientUpdateDone Megginson, et. al. Proposed Standard - Expires: May 2002 11 control optionally attached. If present, the control contains the cookie that represents the current state of the client's data. The value of the reason field of the clientUpdateDone control value will be either lcupClientDisconnect or some lcup error code (not lcupSuccess). The control may not be present if some error occurred. 4.9 Size and Time Limits The search request size or the time limits can only be imposed for non-persistent operations, those that set the updateType field of the ClientUpdateControlValue to synchronizeOnly (for the entire operation) or synchronizeAndPersist (for the initial synchronization phase only). All other operations MUST set both limits to 0. The server SHOULD ignore the limits set for persistent operations. 4.10 Changes vs. Operations The server sends to the client modified entries rather than operations. Given this model, the server communicates a modifyDN operation in one of two ways: by sending the client the current form of the entry (with its new DN) along with an entryUUID attribute, or by sending the client a deletion for the previous DN and an entry for the new DN. The latter method must be used if the server does not support the entryUUID attribute. In either case, if the client state shows that the object that underwent the modifyDN operation was the root of a subtree, the client MUST infer that the DNs of all objects in the subtree have changed such that they reflect the new DN of the subtree root. 4.11 Operations on the Same Connection It is permissible for the client to issue other LDAP operations on the connection used by the protocol. Since each LDAP request/response carries a message id there will be no ambiguity about which PDU belongs to which operation. By sharing the connection among multiple operations, the server will be able to conserve its resources. 4.12 Interactions with Other LDAP Search and Response Controls LCUP defines neither restrictions nor guarantees about the ability to use the LDAP client update control defined in this document in conjunction with other LDAP controls, except for the following: A server MAY ignore non-critical controls supplied with the LCUP control. A server MAY ignore the LCUP control if it is non-critical and it is supplied with other critical controls. If a server receives a critical LCUP control with another critical control, and the server does not support both controls at the same time, the server SHOULD return unavailableCriticalExtension. 5. Additional Features Megginson, et. al. Proposed Standard - Expires: May 2002 12 There are several features present in other protocols or considered useful by clients that are currently not included in the protocol primarily because they are difficult to implement on the server. These features are briefly discussed in this section. This section is intended to open a discussion on the merits of including and approaches to implementing these features. 5.1 Triggered Search Change Type This feature is present in the Triggered Search specification. A flag is attached to each entry returned to the client indicating the reason why this entry is returned. The possible reasons from the draft are "- notChange: the entry existed in the directory and matched the search at the time the operation is being performed, - enteredSet: the entry entered the result, - leftSet: the entry left the result, - modified: the entry was part of the result set, was modified or renamed, and still is in the result set." The leftSet feature is particularly useful because it indicates to the client that an entry is no longer within the client's search specification and the client can remove the associated data from its data store. Ironically, this feature is the hardest to implement on the server because the server does not keep track of the client's state and has no easy way of telling which entries moved out of scope between synchronization sessions with the client. A compromise could be reached by only providing this feature for the operations that occur while the client is connected to the server. This is easier to accomplish because the decision about the change type can be made based only on the change without need for any historical information. This, however, would add complexity to the protocol. 5.2 Persistent Search Change Type This feature is present in the Persistent Search specification. Persistent search has the notion of changeTypes. The client specifies which type of updates will cause entries to be returned, and optionally whether the server tags each returned entry with the type of change that caused that entry to be returned. For LCUP, the intention is full synchronization, not partial. Each entry returned by an LCUP search will have some change associated with it that may concern the client. The client may have to have a local index of entries by DN or unique identifier to determine if the entry has been added or just modified. It is easy for clients to determine if the entry has been deleted because the entryDeleted value of the entryUpdateControl will be TRUE. 5.3 Sending Changes Megginson, et. al. Proposed Standard - Expires: May 2002 13 Some earlier synchronization protocols sent the client(s) only the modified attributes of the entry rather than the entire entry. While this approach can significantly reduce the amount of data returned to the client, it has several disadvantages. First, unless a separate mechanism (like the change type described above) is used to notify the client about entries moving into the search scope, sending only the changes can result in the client having an incomplete version of the data. Let's consider an example. An attribute of an entry is modified. As a result of the change, the entry enters the scope of the client's search. If only the changes are sent, the client would never see the initial data of the entry. Second, this feature is hard to implement since the server might not contain sufficient information to construct the changes based solely on the server's state and the client's cookie. On the other hand, this feature can be easily implemented by the client assuming that the client has the previous version of the data and can perform value by value comparisons. 5.4 Data Size Limits Some earlier synchronization protocols allowed clients to control the amount of data sent to them in the search response. This feature was intended to allow clients with limited resources to process synchronization data in batches. However, an LDAP search operation already provides the means for the client to specify the size limit by setting the sizeLimit field in the SearchRequest to the maximum number of entries the client is willing to receive. While the granularity is not the same, the assumption is that LCUP protocol will be implemented by regular LDAP clients that can deal with the limitations of the LDAP protocol. 5.5 Data Ordering Some earlier synchronization protocols allowed a client to specify that parent entries should be sent before the children for add operations and children entries sent before their parents during delete operations. This ordering helps clients to maintain a hierarchical view of the data in their data store. While possibly useful, this feature is relatively hard to implement and is expensive to perform. 6. Client Side Considerations There are several issues that the implementors of a synchronization client need to consider: - The cookie received from the server after a synchronization session can only be used with the same or more restrictive search specification than the search that generated the cookie. The server will reject the search operation with a cookie that does not satisfy this condition. This is because the client can end up with an incomplete data store otherwise. A more restrictive search specification is the one that generates a subset of the data produced by the original search specification. Megginson, et. al. Proposed Standard - Expires: May 2002 14 - Because an LCUP client specifies the area of the tree with which it wishes to synchronize through the standard LDAP search specification, the client can be returned noSuchObject error if the root of the synchronization area was renamed between the synchronization sessions or during a synchronization session. If this condition occurs, the client can attempt to locate the root by using the root's Unique Identifier saved in client's local data store. It then can repeat the synchronization request using the new search base. In general, a client can detect that an entry was renamed and apply the changes received to the right entry by using the Unique Identifier rather then DN based addressing. - Each active persistent operation requires that an open TCP connection be maintained between an LDAP client and an LDAP server that might not otherwise be kept open. Therefore, client implementors are encouraged to avoid using persistent operations for non-essential tasks and to close idle LDAP connections as soon as practical. The server may close connections if server resources become tight. - The client MAY receive a continuation reference (SearchResultReference [RFC2251 SECTION 4.5.3]) if the search request spans multiple parts of the DIT, some of which may require a different LCUP cookie, some of which may not even be managed by LCUP. The client SHOULD maintain a cache of the LDAP URLs returned in the continuation references and the cookies associated with them. The client is responsible for performing another LCUP search to follow the references, and SHOULD use the cookie corresponding to the LDAP URL for that reference (if it has a cookie). - For alias dereferencing, the server will behave as if the client had requested neverDerefAliases or derefFindingBaseObj as the derefAliases field in the search request [RFC2251, Section 4.5.1]. If the client specifies a value other than neverDerefAliases or derefFindingBaseObj, the server will return protocolError to the client. - Changes to data (e.g., that might affect the LCUP client's filter or scope) or meta-data (e.g., that might affect the client's read access) may affect the presence of entries in the search set. Servers MAY notify LCUP clients of changes to the search set that result from such changes, but an LCUP client MUST NOT assume that such notification will occur. Therefore, in the case where a client is maintaining a cache of entries using LCUP, the data held by the client may be a superset or a subset of the entries that would be returned by a new search request. For example, if access control meta information is changed to deny access to particular entries in the search result set, and the access control information is outside of the search scope (e.g., in a parent entry), the client may have entries stored locally which are no longer part of its desired search set. Similarly, Megginson, et. al. Proposed Standard - Expires: May 2002 15 if entries are added to the search result set due to changes in meta-data, the client's cache of entries may not include these entries. 7. Server Implementation Considerations By design, the protocol supports multiple cookie schemes. This is to allow different implementations the flexibility of storing any information applicable to their environment. A reasonable implementation for an LDUP compliant server would be to use the Replica Update Vector (RUV). For each master, RUV contains the largest CSN seen from this master. In addition, the RUV implemented by some directory servers (not yet in LDUP) contains replica generation - an opaque string that identifies the replica's data store. The replica generation value changes whenever the replica's data is reloaded. Replica generation is intended to signal the replication/synchronization peers that the replica's data was reloaded and that all other replicas need to be reinitialized. RUV satisfies the three most important properties of the cookie: (1) it uniquely identifies the state of client's data, (2) it can be used to synchronize with multiple servers, and (3) it can be used to detect that the server's data was reloaded. A server may support one or more LCUP cookie schemes. It is expected that schemes will be published along with their OIDs as RFCs. If a client initiates an LCUP session with a particular scheme, the server MUST use that same scheme throughout the LCUP session, or until an LCUP context boundary is crossed, in which case the server will usually require a different cookie anyway. In addition, the cookie must contain enough information to allow the server to determine whether the cookie can be safely used with the search specification it is attached to. As discussed earlier in the document, the cookie can only be used with the search specification that is equally or more restrictive than the one for which the cookie was generated. An implementation must make sure that it can correctly update the client's cookie when there is a size limit imposed on the search results by either the client's request or by the server's configuration. If RUV is used as the cookie, entries last modified by a particular master must be sent to the client in the order of their last modified CSN. This ordering guarantees that the RUV can be updated after each entry is sent. The server's DIT may be partitioned into different sections which may have different cookies associated with them. For example, some servers may use some sort of replication mechanism to support LCUP. If so, the DIT may be partitioned into multiple replicas. A client may send an LCUP search request that spans multiple replicas. Some parts of the DIT spanned by the search request scope may be managed by LCUP and some may not. A part of the DIT which is enabled for LCUP is referred to as an LCUP Context. The server SHOULD send a SearchResultReference [RFC2251, SECTION 4.5.3] when the LCUP Context Megginson, et. al. Proposed Standard - Expires: May 2002 16 for a returned entry changes. The server SHOULD return all entries for a particular LCUP Context before returning a reference to other LCUP Contexts or non LCUP enabled parts of the DIT, in order to minimize the processing burden on the clients. The LDAP URL(s) returned MUST contain the DN(s) of the base of another section of the DIT (however the server implementation has partitioned the DIT). The client will then issue another LCUP search using the LDAP URL returned. Each section of the DIT MAY require a different cookie value, so the client SHOULD maintain a cache, mapping the different LDAP URL values to different cookies. If the cookie changes, the scheme may change as well, but the cookie scheme MUST be the same within a given LCUP Context. An implementation SHOULD notify the client about all entries deleted from the search set since the client's last session, but an LCUP client MUST NOT assume that such notification will occur. For example, the server might not notify the client of the deletion of an object if the object left the search set following the client's last synchronization and prior to the object's deletion. An LDUP compliant implementation can achieve this through the use of entry tombstones. The implementation should avoid aggressive tombstone purging since lack of tombstones would cause client's data to be reloaded. We suggest that only the tombstone content be removed during the regular trimming cycle while tombstones themselves are discarded much less frequently. The specification makes no guarantees about how soon a server should send notification of a changed entry to the client when the connection between the client and the server is kept open. This is intentional as any specific maximum delay would be impossible to meet in a distributed directory service implementation. Server implementors are encouraged to minimize the delay before sending notifications to ensure that clients' needs for timeliness of change notification are met. Implementors of servers that support the mechanism described in this document should ensure that their implementation scales well as the number of active persistent operations and the number of changes made in the directory increases. Server implementors are also encouraged to support a large number of client connections if they need to support large numbers of persistent operations. 8. Synchronizing Heterogeneous Data Stores Clients, like a meta directory join engine, synchronizing multiple writable data stores will only work correctly if each piece of information is single mastered (for instance, only by an LDUP compliant directory). This is because different systems have different notions of time and different update resolution procedures. As a result, a change applied on one system can be discarded by the other, thus preventing the data stores from converging. 9. Security Considerations Megginson, et. al. Proposed Standard - Expires: May 2002 17 In some situations, it may be important to prevent general exposure of information about changes that occur in an LDAP server. Therefore, servers that implement the mechanism described in this document SHOULD provide a means to enforce access control on the entries returned and MAY also provide specific access control mechanisms to control the use of the controls and extended operations defined in this document. As with normal LDAP search requests, a malicious client can initiate a large number of persistent search requests in an attempt to consume all available server resources and deny service to legitimate clients. The protocol provides the means to stop malicious clients by disconnecting them from the server. The servers that implement the mechanism SHOULD provide the means to detect the malicious clients. In addition, the servers SHOULD provide the means to limit the number of resources that can be consumed by a single client. Access control on the data can be modified in such a way that the data is no longer visible to the client. The specification does not specify how the server should handle this condition. Moreover, data consistency is not guaranteed if access control is changed from a more restrictive to a less restrictive one. This is because access control can be considered as an additional filter on the search specification and the protocol does not support going from a more to a less restrictive search specification. See Client Side Considerations Section for more detailed explanation of the problem. 10. References [KEYWORDS] S. Bradner, "Keywords for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [RFC2251] M. Wahl, T. Howes, S. Kille "Lightweight Directory Access Protocol", RFC 2251, December 1997. [RFC2252] M. Wahl, A. Coulbeck, T. Howes, S. Kille, "Lightweight Directory Access Protocol (v3): Attribute Syntax Definitions", RFC 2252, December 1997. 11. Acknowledgements The LCUP protocol is based in part on the Persistent Search Change Notification Mechanism defined by Mark Smith, Gordon Good, Tim Howes, and Rob Weltman, the LDAPv3 Triggered Search Control defined by Mark Wahl, and the LDAP Control for Directory Synchronization defined by Michael Armijo. 12. Author's Addresses Rich Megginson Netscape Communications Corp. 901 San Antonio Rd. Megginson, et. al. Proposed Standard - Expires: May 2002 18 Palo Alto, CA 94303-4900 Mail Stop SCA17 - 201 Phone: +1 505 797-7762 Email: richm@netscape.com Olga Natkovich Yahoo, Inc. 701 First Ave. Sunnyvale, CA 94089 Phone: +1 408 349-6153 Email: olgan@yahoo-inc.com Mark Smith Netscape Communications Corp. 901 San Antonio Rd. Palo Alto, CA 94303-4900 Mail Stop SCA17 - 201 Phone: +1 650 937-3477 Email: mcs@netscape.com Jeff Parham Microsoft Corporation One Microsoft Way Redmond, WA 98052-6399 Phone: +1 425 882-8080 Email: jeffparh@microsoft.com 13. Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 14. Appendix A - Summary of Changes Megginson, et. al. Proposed Standard - Expires: May 2002 19 Changes new to version 02: Section 4.2: The lcupCookieScheme operational attribute MUST be present in the root DSE, and MAY be present in entries. Each value of the attribute in the root DSE will be a list of OIDs of cookie schemes followed by the DN of the LCUP context which supports the schemes. The attribute value in the DIT entries will be the list of OIDs followed by the DN of the LCUP context. section 4.5 - the entry uuid is now MAY instead of MUST - if implementers do not wish to identify entries by a unique ID other than DN (which may not be unique), then so be it. For returned SearchResultEntry PDUs other than deleted entries, the client MAY request that the Unique Identifier attribute be returned by specifying it in the attribute list to be returned by the search request. section 4.5 - added "or the base DN of the client's search request." to the phrase. "The server MAY send the entry at the root of the client's tree, or the base DN of the client's search request." I think this clarifies which entry the client may search for. section 4.6 - the clientUpdateDone control is now optional for error conditions. Also, the cookie value of the control is now optional for lcup error conditions (e.g. not lcupSuccess or lcupClientDisconnect). Added section 4.12 - Interactions with Other LDAP Search and Response Controls Added blurb about alias dereferencing back to section 6: "For alias dereferencing, the server will behave as if the client had requested neverDerefAliases or derefFindingBaseObj as the derefAliases field in the search request [RFC2251, Section 4.5.1]. If the client specifies a value other than neverDerefAliases or derefFindingBaseObj, the server will return protocolError to the client." Changed this in section 6: Because an LCUP client specifies the area of the tree with which it wishes to synchronize through the standard LDAP search specification, the client can be returned noSuchObject error if the root of the synchronization area was renamed between the synchronization sessions "or during a synchronization session" Changes new to version 01: The opaque cookie has been split into two parts - a scheme which is an OID, and a value. The value may or may not have a format known to the client, depending on the specified scheme. Section 4.2 describes the new cookie format and defines the LCUP Cookie Value. Megginson, et. al. Proposed Standard - Expires: May 2002 20 Added new section 4.3 - the lcupCookieScheme operational attribute. Changes new to version 00: Added the definition for Unique Identifier (basically copied from the LDUP model doc http://search.ietf.org/internet-drafts/draft- ietf-ldup-model-06.txt. I needed to add the definition here because LCUP needs a Unique Identifier but should not be dependent on LDUP. Removed all normative references to LDUP. I've left the implementation suggestions that refer to LDUP, but LCUP should not be dependent on LDUP. Cleaned up the protocol flows. Removed this text from section 4.8: "Clients MUST NOT issue multiple synchronization requests on the same connection. This is because the protocol includes an extended operation and it would be impossible to decide which synchronization session it belongs to." - This is no longer true, since the extended operation now includes the message ID of the search request. "Client Side Consideration" section - the client will never receive a referral or continuation reference Added section 12. Acknowledgements Removed normative references to documents not depended on. Removed explicit references to software vendors. Section 4.1 - Changed ClientUpdateControlValue to remove the keepConnection and changesOnly fields and replace them with updateType which is an ENUMERATED with three values: synchronizeOnly, synchronizeAndPersist, and persistOnly. Section 4.2 - The EntryUpdateControlValue fields stateUpdate and entryDeleted no longer have DEFAULT values, they must be specified - this eliminates any potential ambiguity. Added this text to the description of the entryDeleted field (section 4.2): "The server SHOULD also set this to TRUE if the entry has left the clients search result set. As far as the client is concerned, a deleted entry is no different than an entry which has left the result set." Section 4.2 - Added an explanation of the concept and requirement for the Unique Identifier. Section 4.4 - Added to the extended operation a request value containing the message id of the operation to stop. Megginson, et. al. Proposed Standard - Expires: May 2002 21 Updated contact information for Olga. Removed Michael Armijo and added Jeff Parham as an author. Changes new to previous version: "Authors" section - added Rich Megginson as the new editor. "Client Side Consideration" section - added a note and a question concerning referral and continuation reference handling. "Client Update Control Value" section (4.1) - clarified the meaning of keepConnection and added a table summarizing the effects of different values of keepConnection and changesOnly. "Stop Client Update Request and Response" - added section 4.4 describing this extended operation. Megginson, et. al. Proposed Standard - Expires: May 2002 22
Datatracker
draft-ietf-ldup-lcup-02
Internet-Draft
