INTERNET-DRAFT Kurt D. Zeilenga
Intended Category: Experimental OpenLDAP Foundation
Expires in six months Jonghyuk Choi
IBM Corporation
18 June 2003
The LDAP Content Synchronization Operation
<draft-zeilenga-ldup-sync-03.txt>
Status of this Memo
This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026.
Distribution of this memo is unlimited. Technical discussion of this
document will take place on the IETF LDUP Working Group mailing list
at <ietf-ldup@imc.org>. Please send editorial comments directly to
the document editor at <Kurt@OpenLDAP.org>.
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''
The list of current Internet-Drafts can be accessed at
<http://www.ietf.org/ietf/1id-abstracts.txt>. The list of
Internet-Draft Shadow Directories can be accessed at
<http://www.ietf.org/shadow.html>.
Copyright (C) The Internet Society (2003). All Rights Reserved.
Please see the Full Copyright section near the end of this document
for more information.
Zeilenga LDAP Content Sync Operation [Page 1]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Abstract
This specification describes the LDAP (Lightweight Directory Access
Protocol) Content Synchronization operation. The operation allows a
client to maintain a shadow copy of a fragment of directory
information tree. It supports both polling for changes and listening
for changes. The operation is defined as an extension of the LDAP
Search operation.
Table of Contents
Status of this Memo 1
Abstract 2
Table of Contents
1. Introduction 3
1.1. Background
1.2. Intended Usage 4
1.3. Overview 5
2. Elements of the Sync Operation 7
2.1. Common ASN.1 elements 8
2.2. Sync Request Control
2.3. Sync State Control 9
2.4. Sync Done Control
2.5. Sync Info Message 10
2.6. Sync Result Codes
3. Content Synchronization
3.1. Synchronization Session
3.2. Content Determination 11
3.3. refreshOnly mode
3.4. refreshAndPersist mode 13
3.5. Search Request Parameters 15
3.6. objectName Issues 16
3.7. Canceling the Sync Operation
3.8. Refresh Required
3.9. Chattiness Considerations 17
3.10. Operation Multiplexing 18
4. Meta Information Considerations
4.1. Entry DN 19
4.2. Operational Attributes
4.3. Collective Attributes
4.4. Access and other administrative controls 20
5. Interaction with other controls
5.1. ManageDsaIT control
5.2. Subentries control 21
6. Shadowing Considerations
7. Security Considerations 22
8. IANA Considerations
Zeilenga LDAP Content Sync Operation [Page 2]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
8.1. Object Identifier
8.2. LDAP Protocol Mechanism 23
8.3. LDAP Result Codes
9. Acknowledgments
10. Normative References 24
11. Informative References 25
12. Authors' Addresses
Appendix A. CSN-based Implementation Considerations 26
Full Copyright 27
1. Introduction
The Lightweight Directory Access Protocol (LDAP) [RFC3377] provides a
mechanism, the search operation [RFC2251], to allow a client to
request the return of content matching a complex set of assertions and
for the server to return this content, subject to access control and
other restrictions, to the client. However, short of repeating a
search operation each time a new copy needed, LDAP does not provide an
effective and efficient mechanism for maintaining synchronized copies
of directory content.
This document defines the LDAP Content Synchronization operation, or
Sync operation for short, which allows a client to maintain a
synchronized shadow copy of a fragment of a Directory Information Tree
(DIT). The Sync operation is defined as a set of controls and other
protocol elements which extend the Search operation.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14 [RFC2119].
1.1. Background
Over the years, a number of directory synchronization approaches have
been suggested. These approaches are inadequate for one or more of
the following reasons:
1) do not ensure a reasonable level of convergence;
2) fail to detect that convergence cannot be achieved (without
reload);
3) require pre-arranged synchronization agreements;
4) require the server to maintain histories of past changes to DIT
content and/or meta information;
5) require the server to maintain synchronization state on a per
client basis; and/or
6) are overly chatty.
Zeilenga LDAP Content Sync Operation [Page 3]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
The Sync operation provides eventual convergence of synchronized
content when possible and, when not, notification that a full reload
is required.
The Sync operation does not require pre-arranged synchronization
agreements.
The Sync operation does not require servers to maintain nor use any
history of past changes to the DIT or to meta information. However,
servers may maintain and use histories (e.g., change logs, tombstones,
DIT snapshots) to reduce the number of messages generated and reduce
their size. As it is not always feasible to maintain and use
histories, the operation may be implemented using purely (current)
state-based approaches.
The Sync operation does not require servers to maintain
synchronization state on a per client basis. However, servers may
maintain and use per client state information to reduce the number of
messages generated and their size.
A synchronization mechanism can be considered overly chatty when
synchronization traffic is not reasonably bounded. The Sync operation
traffic is bounded by the size of updated (or new) entries and the
number of unchanged entries in the content. The operation is designed
to avoid full content exchanges even where the history information
available to the server is insufficient to determine the client's
state.
This document includes a number of non-normative appendices providing
additional information to server implementors.
1.2. Intended Usage
The Sync operation is intended to be used in applications requiring
eventual-convergent content synchronization. Upon completion of each
synchronization stage of the operation, all information to construct a
synchronized shadow copy of the content has been provided to the
client or the client has been notified that a complete content reload
is necessary. Excepting for transient inconsistencies due to
concurrent operation (or other) processing at the server, the shadow
copy is an accurate reflection of the content held by the server.
Each inconsistency is transient in that it will be corrected during
subsequent synchronization requests.
Possible uses include:
- White page service applications may use the Sync operation to
maintain current shadow copy of a DIT fragment. For example, a
Zeilenga LDAP Content Sync Operation [Page 4]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
mail user agent which use the sync operation to maintain a local
copy of an enterprise address book.
- Meta-information engines may use the Sync operation to maintain a
shadow copy of a DIT fragment.
- Caching proxy services may use the Sync operation to maintain a
coherent content cache.
- Lightweight master-slave replication between heterogeneous
directory servers. For example, the Sync operation can be used by
a slave server to maintain a shadow copy of a DIT fragment. (Note:
The International Telephone Union (ITU) has defined the X.500
Directory [X.500] Information Synchronization Protocol (DISP)
[X.525] which may be used for master-slave replication between
LDAP servers. Other experimental LDAP replication protocols also
exist.) The technical specification and applicability statement
of Sync operation based LDAP replication is left to future
documents.
This protocol is not intended to be used in applications requiring
transactional data consistency.
As this protocol transfers all visible values of entries upon change
instead of change deltas, this protocol is not appropriate for
bandwidth-challenged applications or deployments.
1.3. Overview
This section provides an overview of basic ways the Sync operation can
be used to maintain a synchronized shadow copy of a DIT fragment.
- Polling for Changes: refreshOnly mode
- Listening for Changes: refreshAndPersist mode
1.3.1. Polling for Changes (refreshOnly)
To obtain its initial shadow copy, the client issues a Sync request: a
search request with the Sync Request Control with mode set to
refreshOnly. The server, much like it would with a normal search
operation, returns (subject to access controls and other restrictions)
the content matching the search criteria (baseObject, scope, filter,
attributes). Additionally, with each entry returned, the server
provides a Sync State control indicating state add. This control
contains the Universally Unique Identifier (UUID) [UUID] of the entry.
Unlike Distinguished Names (DNs), which may change over time, an
Zeilenga LDAP Content Sync Operation [Page 5]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
entry's UUIDs are stable. The initial content is followed by a
searchResultDone with a Sync Done control. The Sync Done control
provides a syncCookie. The syncCookie represents session state.
To poll for updates to the shadow copy, the client reissues the Sync
operation with the syncCookie previously returned. The server, much
as it would with a normal search operation, determines which content
would be returned as if the operation was a normal search operation.
However, using the syncCookie as an indicator of what content the
client was sent previously, the server sends copies of entries which
have changed with a Sync State control indicating state add. For each
changed entry, all (modified or unmodified) attributes belonging to
the content are sent. For each unchanged entry, the server sends an
empty entry (e.g., no attributes) with a Sync State control indicating
state present. The set of updates is followed by a searchResultDone
with a Sync Done control.
If the server can reliably determine which entries in the prior shadow
copy are no longer present in the content and the number of such
entries is less than or equal to the number of unchanged entries, the
server may, instead of returning an empty entry with state present for
each present entry, send an empty entry with state delete for each
entry which is no longer in the content. Also, the Sync Done control
refreshDeletes is set to TRUE to indicate to the client that this
method was used. This field is FALSE otherwise.
The synchronized shadow copy of the DIT fragment is constructed by the
client.
If refreshDeletes is FALSE, the new copy includes all changed entries
returned by the reissued Sync operation as well as all unchanged
entries identified as being present by the reissued Sync operation,
but whose content is provided by the previous Sync operation. The
unchanged entries not identified as being present are deleted from the
shadow content. They had been either deleted, moved, or otherwise
scoped-out from the content.
If refreshDeletes is TRUE, the new copy includes all changed entries
returned by the reissued Sync operation as well as all other entries
of the previous copy except those which were identified as having been
deleted from the content.
The client can, at some later time, re-poll for changes to this
synchronized shadow copy.
1.3.2. Listening for Changes (refreshAndPersist)
Zeilenga LDAP Content Sync Operation [Page 6]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Polling for changes can be expensive in terms of server, client, and
network resources. The refreshAndPersist mode allows for active
updates of changed entries in the content.
By selecting the refreshAndPersist mode, the client requests the
server to send updates of entries that are changed after the initial
refresh content is determined. Instead of sending a searchResultDone
message as described above, the server sends a Sync Info message to
the client indicating that refresh stage is complete and then enters
persist stage. After receipt of this Sync Info message, the client
will have a synchronized shadow copy as described above.
The server may then send change notifications. For entries to be
added to the returned content, the server sends a searchResultEntry
(with attributes) with a Sync State control indicating state add. For
entries to be deleted from the content, the server sends a
searchResultEntry containing no attributes and a Sync State control
indicating state delete. To modify entries in the return content, the
server sends a searchResultEntry (with attributes) with a Sync State
control indicating state modify. Upon modification of an entry, all
(modified or unmodified) attributes belonging to the content are sent.
Note that renaming an entry of the DIT may cause an add state change
where the entry is renamed into the content, a delete state change
where the entry is renamed out of the content, and a modify state
change where the entry remains in the content. Also note that a
modification of an entry of the DIT may cause an add, delete, or
modify state change to the content.
Upon receipt of a change notification, the client updates its copy of
the content.
If the server desires to update the syncCookie during the persist
stage, it may include the syncCookie in any Sync State control or Sync
Info message returned.
The operation persists until canceled [CANCEL] by the client or
terminated by the server. A Sync Done control may be attached to
searchResultDone message to provide a new syncCookie.
2. Elements of the Sync Operation
The Sync Operation is defined as an extension to the LDAP Search
Operation [RFC2251] where the directory user agent (DUA or client)
submits a SearchRequest message with a Sync Request control and the
directory system agent (DSA or server) responses with zero or more
SearchResultEntry messages, each with a Sync State control; zero or
Zeilenga LDAP Content Sync Operation [Page 7]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
more SearchResultReference messages, each with a Sync State control;
zero or more Sync Intermediate Response messages; and a
searchResultDone message with a Sync Done control.
To allow clients to discover support for this operation, servers
implementing this operation SHOULD publish the IANA-ASSIGNED-OID.1 as
a value of 'supportedControl' attribute [RFC2252] of the root DSA-
specific entry (DSE).
Protocol elements are described using ASN.1 [X.680] with implicit
tags. The term "BER-encoded" means the element is to be encoded using
the Basic Encoding Rules [X.690] under the restrictions detailed in
Section 5.1 of [RFC2251].
2.1 Common ASN.1 elements
2.1.1 syncUUID
The syncUUID is a notational convenience to indicate that, while the
syncUUID type is encoded as an OCTET STRING, its value is restricted
to the string representation of an Universally Unique Identifier
(UUID) defined in [UUID].
syncUUID ::= OCTET STRING
2.1.2 syncCookie
The syncCookie is a notational convenience to indicate that, while the
syncCookie type is encoded as an OCTET STRING, its value is an opaque
value containing information about the synchronization session and its
state. Generally, the session information would include a hash of the
operation parameters which the server requires not be changed and the
synchronization state information includes a commit (log) sequence
number, a change sequence number, or a time stamp.
syncCookie ::= OCTET STRING
2.2 Sync Request Control
The Sync Request Control is an LDAP Control [RFC2251, Section 4.1.2]
where the controlType is the object identifier IANA-ASSIGNED-OID.1 and
the controlValue, an OCTET STRING, contains a BER-encoded
syncRequestValue. The criticality field is either TRUE or FALSE.
syncRequestValue ::= SEQUENCE {
Zeilenga LDAP Content Sync Operation [Page 8]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
mode ENUMERATED {
-- 0 unused
refreshOnly (1),
-- 2 reserved
refreshAndPersist (3)
},
cookie syncCookie OPTIONAL
}
The Sync Request Control is only applicable to the searchRequest
message.
2.3 Sync State Control
The Sync State Control is an LDAP Control [RFC2251, Section 4.1.2]
where the controlType is the object identifier IANA-ASSIGNED-OID.2 and
the controlValue, an OCTET STRING, contains a BER-encoded
syncStateValue. The criticality is FALSE.
syncStateValue ::= SEQUENCE {
state ENUMERATED {
present (0),
add (1),
modify (2),
delete (3)
},
entryUUID syncUUID,
cookie syncCookie OPTIONAL
}
The Sync State Control is only applicable to SearchResultEntry and
SearchResultReference messages.
2.4 Sync Done Control
The Sync Done Control is an LDAP Control [RFC2251, Section 4.1.2]
where the controlType is the object identifier IANA-ASSIGNED-OID.3 and
the controlValue contains a BER-encoded syncDoneValue. The
criticality is FALSE (and hence absent).
syncDoneValue ::= SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDeletes BOOLEAN DEFAULT FALSE
}
The Sync Done Control is only applicable to SearchResultDone message.
Zeilenga LDAP Content Sync Operation [Page 9]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
2.5 Sync Info Message
The Sync Info Message is an LDAP Intermediate Response Message
[LDAPIRM] where responseName is the object identifier
IANA-ASSIGNED-OID.4 and responseValue contains a BER-encoded
syncInfoValue. The criticality is FALSE (and hence absent).
syncInfoValue ::= CHOICE {
newcookie [0] syncCookie,
refreshDone [1] SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDeletes BOOLEAN DEFAULT FALSE
}
}
2.6 Sync Result Codes
The following LDAP resultCode [RFC2251] is defined:
syncRefreshRequired (IANA-ASSIGNED-CODE-0)
3. Content Synchronization
The Sync Operation is invoked by the client sending a searchRequest
message with a Sync Request Control.
The absence of a cookie indicates a request for initial content while
the presence of a cookie indicates a request for content update.
Synchronization Sessions are discussed in Section 3.1. Content
Determination is discussed in Section 3.2.
The mode is either refreshOnly or refreshAndPersist. The refreshOnly
and refreshAndPersist modes are discussed in Section 3.3 and 3.4,
respectively. The refreshOnly mode consists only of a refresh stage,
while the refreshAndPersist mode consists of a refresh stage and a
subsequent persist stage.
3.1. Synchronization Session
A sequence of Sync Operations where the last cookie returned by the
server for one operation is provided by the client in the next
operation are said to belong to the same Synchronization Session.
The client MUST specify the same content controlling parameters (see
Section 3.5) in each Search Request of the session. The client SHOULD
Zeilenga LDAP Content Sync Operation [Page 10]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
also issue each Sync request of a session under the same
authentication and authorization associations with equivalent
integrity and confidential protections. If the server does not
recognize the request cookie or the request is made under different
associations or inequivalent protections, the server SHALL process the
request as if no cookie had been provided.
A Synchronization Session may span multiple LDAP sessions between the
client and the server. The client SHOULD issue each Sync request of a
session to the same server. Note: Shadowing considerations are
discussed in Section 6.
3.2. Content Determination
The content to be provided is determined by parameters of the Search
Request, as described in [RFC2251], and possibly other controls. The
same content SHOULD be used in each Sync request of a session. If
different content is requested and the server is unwilling or unable
to process the request, the server SHALL process the request as if no
cookie had been provided.
The content may not necessarily include all entries or references
which would be returned by a normal search operation nor, for those
entries included, not all attributes returned by a normal search.
When the server is unwilling or unable to provide synchronization for
any attribute for a set of entries, the server MUST treat all filter
components matching against these attributes as Undefined and MUST NOT
return these attributes in searchResultEntry responses.
Servers SHOULD support synchronization for all non-collective
user-applications attributes for all entries.
The server may also return continuation references to other servers or
to itself. The latter is allowed as the server may partition the
entries it holds into separate synchronization contexts.
The client may chase all or some of these continuations, each as a
separate content synchronization session.
3.3. refreshOnly mode
A Sync request with mode refreshOnly and no cookie is a poll for
initial content. A Sync request with mode refreshOnly and cookie is a
poll for content update.
Zeilenga LDAP Content Sync Operation [Page 11]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
3.3.1. Initial Content Poll
Upon receipt of the request, the server provides the initial content
using a set of zero or more searchResultEntry and
searchResultReference messages followed by a searchResultDone message.
Each searchResultEntry message SHALL include a Sync State control of
state add, entryUUID containing the entry's UUID, and no cookie. Each
searchResultReference message SHALL include a Sync State control of
state add, entryUUID containing the UUID associated with the reference
(normally the UUID of the associated referral [RFC3296] object), and
no cookie. The searchResultDone message SHALL include a Sync Done
control. The refreshDeletes SHALL be FALSE.
A resultCode value of success indicates the operation successfully
completed. Otherwise, the result code indicates the nature of
failure.
If the operation is successful, a cookie SHOULD be returned for use in
subsequent Sync operations.
3.3.2. Content Update Poll
Upon receipt of the request the server provides the content refresh
using a set of zero or more searchResultEntry and
searchResultReference messages followed by a searchResultDone message.
The server is REQUIRED to either:
a) provide the sequence of messages necessary for eventual
convergence of the client's copy of the content to the server's
copy,
b) treat the request as an initial content request (e.g., ignore
the cookie),
c) indicate that convergence is not possible by returning
syncRefreshRequired,
d) return a resultCode other than success or syncRefreshRequired.
For each entry or reference added to the content or was changed since
the previous Sync operation indicated by the cookie, the server
returns a searchResultEntry or searchResultReference message,
respectively, each with a Sync State cookie of state add, entryUUID
containing the UUID of the entry or reference, and no cookie. Each
searchResultEntry message represents the current state of a changed
entry. Each SearchResultReference message represents the current
Zeilenga LDAP Content Sync Operation [Page 12]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
state of a changed reference.
For each entry which has not been changed since the previous Sync
operation, a searchResultEntry is returned whose objectName reflects
the entry's current DN, the attributes field is empty, and a Sync
State control of state present, entryUUID containing the UUID of the
entry, and no cookie. For each reference which has not been changed
since the previous Sync operation, a searchResultReference containing
an empty SEQUENCE OF LDAPURL is returned with a Sync State control of
state present, entryUUID containing the UUID of the entry, and no
cookie. No messages are sent for entries or references which are no
longer in content.
As an alternative to sending messages for each entry and reference
which has not been changed, the server may instead return the
following. For each entry no longer in content, return a
searchResultEntry whose objectName reflects a past DN of the entry or
is empty, the attributes field is empty, and a Sync State control of
state delete, entryUUID containing the UUID of the deleted entry, and
no cookie. For each reference no longer in content, a
searchResultReference containing an empty SEQUENCE OF LDAPURL is
returned with a a Sync State control of state delete, entryUUID
containing the UUID of the deleted reference, and no cookie.
A resultCode value of success indicates the operation successfully
completed. A resultCode value of syncRefreshRequired indicates that a
full or partial refresh is needed. Otherwise, the result code
indicates the nature of failure. A cookie is provided in the Sync
Done Control for use in subsequent Sync operations when incremental
sychronization is possible.
3.4. refreshAndPersist mode
A Sync request with mode refreshAndPersist asks for initial content or
content update (during the refresh stage) followed by change
notifications (during the persist stage).
3.4.1. refresh stage
The content refresh is provided as described in Section 3.3 excepting
that successful completion of content refresh is indicated by sending
a Sync Info message with a refreshDone value instead of a
SearchResultDone message with resultCode success. A cookie SHOULD be
returned in the refreshDone value for use in subsequent Sync
operations.
Zeilenga LDAP Content Sync Operation [Page 13]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
3.4.2. persist stage
Change notifications are provided during the persist stage.
As updates are made to the DIT the server notifies the client of
changes to the content. DIT updates may cause entries references to
be added to the content, deleted from the content, or modify entries
in the content. DIT updates may also cause references to be added,
deleted, or modified within the content.
Where DIT updates cause an entry to be added to the content, the
server provides a searchResultEntry message which represents the entry
as it appears in the content. The message SHALL include a Sync State
control with state of add, entryUUID containing the entry's UUID, and
an optional cookie.
Where DIT updates cause a reference to be added to the content, the
server provides a searchResultReference message which represents the
reference in the content. The message SHALL include a Sync State
control with state of add, entryUUID containing the UUID associated
with the reference, and an optional cookie.
Where DIT updates cause an entry to be modified in the content, the
server provides a searchResultEntry message which represents the entry
as it appears in the content. The message SHALL include a Sync State
control with state of modify, entryUUID containing the entry's UUID,
and an optional cookie.
Where DIT updates cause a reference to be modified in the content, the
server provides a searchResultEntry message which represents the
reference in the content. The message SHALL include a Sync State
control with state of modify, entryUUID containing the UUID associated
with the reference, and an optional cookie.
Where DIT updates cause an entry to be deleted from the content, the
server provides a searchResultReference message with an empty SEQUENCE
OF LDAPURL. The message SHALL include a Sync State control with state
of delete, entryUUID containing the UUID associated with the
reference, and an optional cookie.
Where DIT updates cause a reference to be deleted from the content,
the server provides a searchResultEntry message with no attributes.
The message SHALL include a Sync State control with state of delete,
entryUUID containing the entry's UUID, and an optional cookie.
With each of these messages, the server may provide a new cookie to be
used in subsequent Sync operations. Additionally, the server may also
return Sync Info messages of choice newCookie to provide a new cookie.
Zeilenga LDAP Content Sync Operation [Page 14]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
The client SHOULD use the newest (last) cookie it received from the
server in subsequent Sync operations.
3.5. Search Request Parameters
As stated in Section 3.1, the client SHOULD specify the same content
controlling parameters in each Search Request of the session. All
fields of the SearchRequest message are considered content controlling
parameters except for sizeLimit and timeLimit.
3.5.1. baseObject Issues
As with the normal search operation, the refresh and persist stages
are not isolated from DIT changes. It is possible that the entry
referred to be the baseObject be deleted, renamed, or moved. It is
also possible that alias object used in finding the entry referred to
by the baseObject is changed such that the baseObject refers to a
different entry.
If the DIT is updated during processing of the Sync Operation in a
manner that causes the baseObject to no longer refers to any entry or
changes which entry the baseObject refers to, the server SHALL return
an appropriate non-success result code such as noSuchObject,
aliasProblem, aliasDereferencingProblem, referral, or
syncRefreshRequired.
3.5.2. derefAliases Issues
This operation does not support alias dereferencing during searching.
The client SHALL specify neverDerefAliases or derefFindingBaseObj for
the searchRequest derefAliases parameter. The server SHALL treat
other values (e.g., derefInSearching, derefAlways) as protocol errors.
3.5.3. sizeLimit Issues
The sizeLimit applies only to entries (regardless of their syncState)
returned during refreshOnly processing or the refresh stage of the
refreshAndPersist processing.
3.5.4. timeLimit Issues
For a refreshOnly Sync operation, the timeLimit applies to the whole
operation. For a refreshAndPersist operation, the timeLimit applies
Zeilenga LDAP Content Sync Operation [Page 15]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
to processing up to and including generating the Sync Info message
with a refreshDone value.
3.5.5. filter Issues
The client SHOULD avoid filter assertions which apply to values of
attributes likely to be considered by the server as holding meta-
information. See Section 4.
3.6. objectName Issues
The Sync operation uses entryUUID values provided in the Sync State
control as the primary keys to entries. The client MUST use these
entryUUIDs to correlate synchronization messages.
In some circumstances the DN returned may not reflect the entry's
current DN. In particular, when the entry is being deleted from the
content, the server may provide an empty DN if the server does not
wish to disclose the entry's current DN (or, if deleted from the DIT,
the entry's last DN).
It should also be noted that the entry's DN may be viewed as meta
information (see Section 4.1).
3.7. Canceling the Sync Operation
Servers SHOULD implement the LDAP Cancel [CANCEL] operation and
support cancellation of outstanding Sync operations as described here.
To cancel an outstanding Sync Operation, the client SHOULD issue a
LDAP Cancel [CANCEL] operation.
If at any time the server becomes unwilling or unable to continue
processing a Sync Operation, the server SHALL return a
searchResultDone with a non-success resultCode indicating the reason
for the termination of the operation.
Whether the client or server initiated the termination, the server may
provide a cookie in the Sync Done control for use in subsequent Sync
operations.
3.8. Refresh Required
Zeilenga LDAP Content Sync Operation [Page 16]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
In order to achieve the eventual-convergent synchronization, the
server may terminate the Sync operation in refresh or persist stage by
returning a syncRefreshRequired resultCode to the client. If a cookie
is not provided in this response, a full refresh is needed. If a
cookie is provided in this response, an incremental refresh is needed.
To obtain a full refresh, the client then issues a new synchronization
request with no cookie. To obtain an incremental reload, the client
issues a new synchronization with the provided cookie.
The server may choose to provide a full copy in the refresh stage
(e.g., ignore the cookie) instead of providing an incremental refresh
in order to achieve the eventual convergence.
In the case of persist stage Sync, the server returns the resultCode
of syncRefreshRequired to the client to indicate that the client needs
to issue a new Sync operation in order to obtain a synchronized copy
of the content. If a cookie is not provided, a full refresh is needed.
If a cookie is provided, an incremental refresh is needed.
The server may also return syncRefreshRequired if it determines that a
refresh would be more efficient than sending all the messages required
for convergence.
It is noted that the client may receive one or more of
searchResultEntry, searchResultReference, and/or Sync Info messages
before it receives searchResultDone message with the
syncRefreshRequired result code.
3.9. Chattiness Considerations
The server MUST ensure that the number of entry messages generated to
refresh the client content does not exceed the number of entries
presently in the content. While there is no requirement for servers
to maintain historical information, if the server has sufficient
history to allow it to reliably determine which entries in the prior
shadow copy are no longer present in the content and the number of
such entries is less than or equal to the number of unchanged entries,
the server SHOULD generate delete entry messages instead of present
entry messages (see Section 3.3.2).
The server SHOULD maintain enough (current or historical) state
information (such as a context-wide last modify time stamp), to
determine that no changes were made in the context since the content
to refresh was provided and, and when no changes were made, generate
zero delete entry messages instead of present messages.
Zeilenga LDAP Content Sync Operation [Page 17]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
The server implementor should also consider chattiness issues which
span multiple Sync operations of a session. As noted in Section 3.8,
the server may return syncRefreshRequired if it determines that a
refresh would be more efficient than continuing under the current
operation.
The server SHOULD transfer a new cookie frequently to avoid having to
transfer information already provided to the client. Even where DIT
changes do not cause content synchronization changes to be
transferred, it may be advantageous to provide a new cookie using a
Sync Info message. However, the server SHOULD avoid overloading the
client or network with Sync Info messages.
During persist mode, the server SHOULD coalesce multiple outstanding
messages updating the same entry. The server MAY delay generation of
an entry update in anticipation of subsequent changes to that entry
which could be coalesced. The length of the delay should be long
enough to allow coalescing of update requests issued back to back but
short enough that the transient inconsistency induced by the delay is
corrected in a timely manner.
It is also noted that there may be many clients interested in a
particular directory change, and servers attempting to service all of
these at once may cause congestion on the network. The congestion
issues are magnified when the change requires a large transfers to
each interested client. Implementors and deployers of servers should
take steps prevent and manage network congestion.
3.10. Operation Multiplexing
The LDAP protocol model [RFC2251] allows operations to be multiplexed
over a single LDAP session. Clients SHOULD NOT maintain multiple LDAP
sessions with the same server. Servers SHOULD ensure that responses
from concurrently processed operations are interleaved fairly.
Clients SHOULD combine Sync operations whose result set is largely
overlapping. This avoids having to return multiple messages, once for
each overlapping session, for changes to entries in the overlap.
Clients SHOULD NOT combine Sync operations which are largely
non-overlapping result sets. This ensures that an event requiring an
syncRefreshRequired response can be limited to as few as possible
result sets.
4. Meta Information Considerations
Zeilenga LDAP Content Sync Operation [Page 18]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
4.1. Entry DN
As an entry's DN is constructed from its relative DN (RDN) and the
entry's parent's DN, it is often viewed as meta information.
While renaming or moving a superior to an entry causes the entry's DN
to change, that change SHOULD NOT, by itself, cause synchronization
message to be sent for that entry. However, if renaming or moving of
a superior could cause the entry to be added or deleted from the
content and, if so, appropriate synchronization messages should be
generated to indicate this to the client.
Where a server treats the entry's DN as meta information, the server
SHALL either
- evaluate all MatchingRuleAssertions [RFC2251] to TRUE if
matching a value of an attribute of the entry and otherwise
Undefined, or
- evaluate all MatchingRuleAssertion with dnAttributes of TRUE as
Undefined.
The latter choice is offered for ease of server implementation.
4.2. Operational Attributes
Where values of an operational attribute is determined by values not
held as part of the entry it appears in, the operational attribute
SHOULD NOT support synchronization of that operational attribute.
For example, in servers which implement X.501 subschema model [X.501],
servers should not support synchronization of the subschemaSubentry
attribute as its value is determined by values held and administrated
in subschema subentries.
For a counter example, servers which implement aliases
[RFC2256][X.501] can support synchronization of the aliasedObjectName
attribute as its values are held and administrated as part of the
alias entries.
Servers SHOULD support synchronization of the following operational
attributes: createTimestamp, modifyTimestamp, creatorsName,
modifiersName [RFC2252]. Servers MAY support synchronization of other
operational attributes.
4.3. Collective Attributes
Zeilenga LDAP Content Sync Operation [Page 19]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
A collective attribute is "a user attribute whose values are the same
for each member of an entry collection" [X.501]. Use of collective
attributes in LDAP is discussed in [COLLECTIVE].
Modification of a collective attribute generally affects the content
of multiple entries, each a member of the collection. It is
inefficient to include values of collective attributes visible in
entries of the collection, as a single modification of a collective
attribute requires transmission of multiple SearchResultEntry (one of
each entry of the collection which the modification affected) to be
transmitted.
Servers SHOULD NOT synchronize collective attributes appearing in
entries of any collection. Servers MAY support synchronization of
collective attributes appearing in collective attribute subentries.
4.4. Access and other administrative controls
Entries are commonly subject to access and other administrative
controls. While portions of the policy information governing a
particular entry may be held in the entry, policy information is often
held elsewhere (in superior entries, in subentries, in the root DSE,
in configuration files, ...). Because of this, changes to policy
information make it difficult to ensure eventual convergence during
incremental synchronization.
Where it is impractical or infeasible to generate content changes
resulting from a change to policy information, servers may opt to
return syncRefreshRequired or treat the Sync Operation as an initial
content request (e.g., ignore the cookie).
5. Interaction with other controls
The Sync Operation may be used with:
- ManageDsaIT Control [RFC3296]
- Subentries Control [SUBENTRY]
as described below. The Sync operation may be used with other LDAP
extensions as detailed in other documents.
5.1. ManageDsaIT control
The ManageDsaIT control [RFC3296] indicates that the operation acts
upon the DSA Information Tree and causes referral and other special
Zeilenga LDAP Content Sync Operation [Page 20]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
objects to be treated as normal objects with respect to the operation.
5.2. Subentries control
The Subentries control is used with the search operation "to control
the visibility of entries and subentries which are within scope"
[SUBENTRY]. When used with the Sync Operation, the subentries control
and other factors (search scope, filter, etc.) are used to determining
whether an entry or subentry appear in the content or not.
6. Shadowing Considerations
As noted in [RFC2251], some servers may hold shadow copies of entries
which can be used to answer search and comparison queries. Such
servers may also support content synchronization requests. This
section discusses considerations for implementors and deployers for
the implementation and deployment of the Sync operation in shadowed
directories.
While a client may know of multiple servers which are equally capable
of being used to obtain particular directory content from, a client
SHOULD NOT assume that each of these server is equally capable of
continuing a content synchronization session. As stated in Section
3.1, the client SHOULD issue each Sync request of a Sync session to
the same server.
However, through domain naming or IP address redirection or other
technics, multiple physical servers can be made to appear as one
logical server to a client. Only servers which are equally capable in
regards to their support for the Sync operation and which hold equally
complete copies of the entries should be made to appear as one logical
server. In particular, each physical server acting as one logical
server SHOULD be equally capable of continuing a content
synchronization based upon cookies provided any of the other physical
servers without requiring a full reload. Because there is no standard
LDAP shadowing mechanism, the specification of how to independently
implement equally capable servers (as well as the precise definition
of "equally capable") is left to future documents.
It is noted that it may be difficult for the server to reliably
determine what content was provided to the client by another server,
especially in the shadowing environments which allow shadowing events
to be coalesced. Where so, the content poll alternative approach
(sending of entry delete messages instead of entry present messages)
discussed in Section 3.3.2 may not be applicable.
Zeilenga LDAP Content Sync Operation [Page 21]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
7. Security Considerations
In order to maintain a synchronized copy of the content, a client is
to delete information from its copy of the content as described above.
However, the client may maintain knowledge of information disclosed to
it by the server separate from its copy of the content used for
synchronization. Management of this knowledge is beyond the scope of
this document.
While the information provided by a series of refreshOnly Sync
operations is similar to that provided by a series of Search
operations, persist stage may disclose additional information. A
client may be able to discern information about the particular
sequence of update operations which caused content change.
Implementors should take precautions against malicious cookie content,
including malformed cookies or valid cookies used with different
security associations and/or protections in attempt to obtain
unauthorized access to information. Servers may include a digital
signature in the cookie to detect tampering.
The operation may be the target of direct denial of service attacks.
Implementors should provide safeguards to ensure the operation is not
abused. Servers may place access control or other restrictions upon
the use of this operation.
It is noted that even small updates to the directory may cause
significant amount of traffic to be generated to clients using this
operation. An user could abuse its update priviledges to mount an
indirect denial of service to these clients, other clients, and/or
portions of the network. Servers should provide safeguards to ensure
update operations are not abused.
Implementors of this (or any) LDAP extension should be familiar with
general LDAP security considerations [RFC3377].
8. IANA Considerations
Registration of the following values is requested.
8.1. Object Identifier
It is requested that IANA register an LDAP Object Identifier to
identify elements of the LDAP Content Synchronization Operation as
defined in this document.
Zeilenga LDAP Content Sync Operation [Page 22]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Subject: Request for LDAP Object Identifier Registration
Person & email address to contact for further information:
Kurt Zeilenga <kurt@OpenLDAP.org>
Specification: RFC XXXX
Author/Change Controller: IESG
Comments:
Identifies elements of the LDAP Content Synchronization Operation
8.2. LDAP Protocol Mechanism
It is requested that IANA register the LDAP Protocol Mechanism
described in this document.
Subject: Request for LDAP Protocol Mechanism Registration
Object Identifier: IANA-ASSIGNED-OID
Description: LDAP Content Synchronization Control
Person & email address to contact for further information:
Kurt Zeilenga <kurt@openldap.org>
Usage: Control
Specification: RFC XXXX
Author/Change Controller: IESG
Comments: none
8.3. LDAP Result Codes
It is requested that IANA register the LDAP Result Codes described in
this document.
Subject: LDAP Result Code Registration
Person & email address to contact for further information:
Kurt Zeilenga <kurt@OpenLDAP.org>
Result Code Name: syncRefreshRequired (IANA-ASSIGNED-CODE-0)
Specification: RFC XXXX
Author/Change Controller: IESG
Comments: none
9. Acknowledgments
This document borrows significantly from the LDAP Client Update
Protocol [LCUP] work. This document also benefited from Persistent
Search [PSEARCH], Triggered Search [TSEARCH], and Directory
Synchronization [DIRSYNC] works. This document also borrows from
"Lightweight Directory Access Protocol (v3)" [RFC2251].
Zeilenga LDAP Content Sync Operation [Page 23]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
10. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14 (also RFC 2119), March 1997.
[RFC2251] Wahl, M., T. Howes and S. Kille, "Lightweight Directory
Access Protocol (v3)", RFC 2251, December 1997.
[RFC2252] Wahl, M., A. Coulbeck, T. Howes, and S. Kille,
"Lightweight Directory Access Protocol (v3): Attribute
Syntax Definitions", RFC 2252, December 1997.
[RFC3296] Zeilenga, K., "Named Subordinate References in
Lightweight Directory Access Protocol (LDAP)
Directories", RFC 3296, July 2002.
[RFC3377] Hodges, J. and R. Morgan, "Lightweight Directory Access
Protocol (v3): Technical Specification", RFC 3377,
September 2002.
[LDAPIRM] Harrison, R. and K. Zeilenga, "LDAP Intermediate
Response",
draft-rharrison-ldap-intermediate-resp-00.txt, a work in
progress.
[SUBENTRY] Zeilenga, K. and S. Legg, "Subentries in LDAP",
draft-zeilenga-ldap-subentry-xx.txt, a work in progress.
[X.680] International Telecommunication Union -
Telecommunication Standardization Sector, "Abstract
Syntax Notation One (ASN.1) - Specification of Basic
Notation", X.680(1997) (also ISO/IEC 8824-1:1998).
[X.690] International Telecommunication Union -
Telecommunication Standardization Sector, "Specification
of ASN.1 encoding rules: Basic Encoding Rules (BER),
Canonical Encoding Rules (CER), and Distinguished
Encoding Rules (DER)", X.690(1997) (also ISO/IEC
8825-1:1998).
[CANCEL] Zeilenga, K., "LDAP Cancel Extended Operation",
draft-zeilenga-ldap-cancel-xx.txt, a work in progress.
[COLLECTIVE] Zeilenga, K., "Collective Attributes in LDAP",
draft-zeilenga-ldap-collective-xx.txt, a work in
progress.
[UUID] International Organization for Standardization (ISO),
"Information technology - Open Systems Interconnection -
Zeilenga LDAP Content Sync Operation [Page 24]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Remote Procedure Call", ISO/IEC 11578:1996
11. Informative References
[RFC2256] Wahl, M., "A Summary of the X.500(96) User Schema for
use with LDAPv3", RFC 2256, December 1997.
[RFC3383] Zeilenga, K., "IANA Considerations for LDAP", BCP 64
(also RFC 3383), September 2002.
[X.500] International Telecommunication Union -
Telecommunication Standardization Sector, "The Directory
-- Overview of concepts, models and services,"
X.500(1993) (also ISO/IEC 9594-1:1994).
[X.511] International Telecommunication Union -
Telecommunication Standardization Sector, "The
Directory: Abstract Service Definition", X.511(1993).
[X.525] International Telecommunication Union -
Telecommunication Standardization Sector, "The
Directory: Replication", X.525(1993).
[DIRSYNC] Armijo, M., "Microsoft LDAP Control for Directory
Synchronization", draft-armijo-ldap-dirsync-xx.txt, a
work in progress.
[LCUP] Megginson, R., et. al., "LDAP Client Update Protocol",
draft-ietf-ldup-lcup-xx.txt, a work in progress.
[PSEARCH] Smith, M., et. al., "Persistent Search: A Simple LDAP
Change Notification Mechanism",
draft-ietf-ldapext-psearch-xx.txt, a work in progress.
[TSEARCH] Wahl, M., "LDAPv3 Triggered Search Control",
draft-ietf-ldapext-trigger-xx.txt, a work in progress.
12. Authors' Addresses
Kurt D. Zeilenga
OpenLDAP Foundation
<Kurt@OpenLDAP.org>
Jonghyuk Choi
IBM Corporation
<jongchoi@us.ibm.com>
Zeilenga LDAP Content Sync Operation [Page 25]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Appendix A. CSN-based Implementation Considerations
This appendix is provided for informational purposes only, it is not a
normative part of the LDAP Content Synchronization Operation's
technical specification.
This appendix discusses LDAP Content Synchronization Operation server
implementation considerations associated with a Change Sequence Number
based approaches.
Change Sequence Number based approaches are targetted for use in
servers which do not maintain historical information (e.g., change
logs, state snapshots, etc.) about changes made to the Directory and
hence, must rely on current directory state and minimal
synchronization state information embedded in Sync Cookie. Servers
which maintain historical information should consider other approaches
which exploit the historical information.
A Change Sequence Number is, effectively a time stamp has sufficient
granularity to ensure that relationship in time of two updates to the
same object can be determined. Change Sequence Numbers are not to be
confused with Commit Sequence Numbers or Commit Log Record Numbers. A
Commit Sequence Number allow one to determine how to two commits (to
the same object or different objects) relate to each other in time.
Change Sequence Number associated with different entries may be
committed out of order. In the remainder of this Appendix, the term
CSN refers to a Change Sequence Number.
In these approaches, the server not only maintains a CSN for each
directory entry (the entry CSN), but maintains a value which we will
call the context CSN. The context CSN is the greatest committed entry
CSN which is not greater than any outstanding entry CSNs for all
entries in a directory context. The values of context CSN are used in
syncCookie values as synchronization state indicators.
As search operations are not isolated from individual directory update
operations and individual update operations cannot be assumed to be
serialized, one cannot assume that the returned content incorporates
all relevant changes whose change sequence number is less than or
equal to the greatest entry CSN in the content. The content
incorporates all the relevant changes whose change sequence number is
less than or equal to context CSN before search processing. The
content may also incorporate any subset of the the changes whose
change sequence number is greater than context CSN before search
processing but less than or equal to the context CSN after search
processing. The content does not incorporate any of the changes whose
CSN is greater than the context CSN after search processing.
Zeilenga LDAP Content Sync Operation [Page 26]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
A simple server implementation could use value of the context CSN
before search processing to indicate state. Such an implementation
would embed this value into each SyncCookie returned. We'll call this
the cookie CSN. When a refresh was requested, the server would simply
entry "update" messages for all entries in the content whose CSN is
greater than the cookie CSN and entry "present" messages for all other
entries in the content. However, if the current context CSN is same
as the cookie CSN, the server should instead generate zero "updates",
zero "delete" messages and indicate refreshDeletes of TRUE as the
directory has not changed.
The implementation should also consider the impact of changes to meta
information, such as access controls, which affects content
determination. One approach is for the server to maintain a context
wide meta information CSN or meta CSN. This meta CSN would be updated
whenever meta information affecting content determination was changed.
If the value of the meta CSN is greater than cookie CSN, the server
should ignore the cookie and treat the request as an initial request
for content.
Additionally, servers may want to consider maintaining some
per-session historical state information to reduce the number of
messages in needed to be transferred during incremental refreshes.
Specifically, a server could record information about entries as they
leave the scope of a disconnected sync session and later use this
information to generate "update"+"delete" messages instead of
"update"+"present" messages. When there are large number of sessions,
it may only make sense to maintain such history for clients where it
of most value. Also, servers taking this approach need to consider
resource consumption issues to ensure reasonable server operation and
to protect against abuse. It may be appropriate to restrict this mode
of operation by policy.
Full Copyright
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implmentation may be prepared, copied, published and
distributed, in whole or in part, without restriction of any kind,
provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Zeilenga LDAP Content Sync Operation [Page 27]
INTERNET-DRAFT draft-zeilenga-ldup-sync-03 18 June 2003
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be followed,
or as required to translate it into languages other than English.
Zeilenga LDAP Content Sync Operation [Page 28]