Commit 0525d184 authored by Kurt Zeilenga's avatar Kurt Zeilenga
Browse files

Latest LDAP Sync I-D (this revision was submitted to the RFC Editor

for publication)
parent 4ac04744
INTERNET-DRAFT Kurt D. Zeilenga
Intended Category: Standard Track OpenLDAP Foundation
Expires in six months Jonghyuk Choi
Intended Category: Experimental OpenLDAP Foundation
Expires in six months Jong Hyuk Choi
IBM Corporation
5 May 2003
3 February 2004
LDAP Content Synchronization Operation
<draft-zeilenga-ldup-sync-02.txt>
The LDAP Content Synchronization Operation
<draft-zeilenga-ldup-sync-05.txt>
1. Status of this Memo
Status of this Memo
This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026.
......@@ -43,10 +39,10 @@ Expires in six months Jonghyuk Choi
Internet-Draft Shadow Directories can be accessed at
<http://www.ietf.org/shadow.html>.
Copyright 2003, The Internet Society. All Rights Reserved.
Copyright (C) The Internet Society (2004). All Rights Reserved.
Please see the Copyright section near the end of this document for
more information.
Please see the Full Copyright section near the end of this document
for more information.
......@@ -57,154 +53,214 @@ Expires in six months Jonghyuk Choi
Zeilenga LDAP Content Sync Operation [Page 1]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
Abstract
This specification describes the LDAP (Lightweight Directory Access
Protocol) Content Synchronization operation. The operation allows a
client to maintain a shadow copy of a fragment of directory
information tree. It supports both polling for changes and listening
for changes. The operation is defined as an extension of the LDAP
Search operation.
Protocol) Content Synchronization Operation. The operation allows a
client to maintain a copy of a fragment of directory information tree.
It supports both polling for changes and listening for changes. The
operation is defined as an extension of the LDAP Search Operation.
Table of Contents
Status of this Memo 1
Abstract 2
Table of Contents
1. Introduction 3
1.1. Background
1.2. Intended Usage 4
1.3. Overview 5
1.4. Conventions
2. Elements of the Sync Operation 8
2.1. Common ASN.1 Elements 9
2.2. Sync Request Control
2.3. Sync State Control
2.4. Sync Done Control 10
2.5. Sync Info Message
2.6. Sync Result Codes 11
3. Content Synchronization
3.1. Synchronization Session
3.2. Content Determination 12
3.3. refreshOnly Mode 13
3.4. refreshAndPersist Mode 16
3.5. Search Request Parameters 17
3.6. objectName Issues 18
3.7. Canceling the Sync Operation 19
3.8. Refresh Required
3.9. Chattiness Considerations 20
3.10. Operation Multiplexing 21
4. Meta Information Considerations 22
4.1. Entry DN
4.2. Operational Attributes
4.3. Collective Attributes 23
4.4. Access and Other Administrative Controls
5. Interaction with Other Controls
5.1. ManageDsaIT Control 24
5.2. Subentries Control
6. Shadowing Considerations
7. Security Considerations 25
8. IANA Considerations
Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14 [RFC2119].
Zeilenga LDAP Content Sync Operation [Page 2]
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
Protocol elements are described using ASN.1 [X.680]. The term
"BER-encoded" means the element is to be encoded using the Basic
Encoding Rules [X.690] under the restrictions detailed in Section 5.1
of [RFC2251].
8.1. Object Identifier 26
8.2. LDAP Protocol Mechanism
8.3. LDAP Result Codes
9. Acknowledgments
10. Normative References 27
11. Informative References 28
12. Authors' Addresses 29
Appendix A. CSN-based Implementation Considerations
Intellectual Property Rights 31
Full Copyright
1. Introduction
The Lightweight Directory Access Protocol (LDAP) [RFC3377] provides a
mechanism, the search operation [RFC2251], to allow a client to
request the return of content matching a complex set of assertions and
for the server to return this content, subject to access control and
other restrictions, to the client. However, short of repeating a
search operation each time a new copy needed, LDAP does not provide an
effective and efficient mechanism for maintaining synchronized copies
of directory content.
This document defines the LDAP Content Synchronization operation, or
Sync operation for short, which allows a client to maintain a
synchronized shadow copy of a fragment of a Directory Information Tree
(DIT). The Sync operation is defined as a set of controls and other
protocol elements which extend the Search operation.
mechanism, the search operation [RFC2251], which allows a client to
request directory content matching a complex set of assertions and for
the server to return this content, subject to access control and other
restrictions, to the client. However, LDAP does not provide (despite
the introduction of numerous extensions in this area) an effective and
efficient mechanism for maintaining synchronized copies of directory
content. This document introduces a new mechanism specifically
designed to met the content synchronization requirements of
sophisticated directory applications.
This document defines the LDAP Content Synchronization Operation, or
Sync Operation for short, which allows a client to maintain a
synchronized copy of a fragment of a Directory Information Tree (DIT).
The Sync Operation is defined as a set of controls and other protocol
elements which extend the Search Operation.
1.1. Background
Over the years, a number of directory synchronization approaches have
been suggested. These approaches are inadequate for one or more of
the following reasons:
Over the years, a number of content synchronization approaches have
been suggested for use in LDAP directory services. These approaches
are inadequate for one or more of the following reasons:
1) do not ensure a reasonable level of convergence;
2) fail to detect that convergence cannot be achieved (without
- fail to ensure a reasonable level of convergence;
- fail to detect that convergence cannot be achieved (without
reload);
- require pre-arranged synchronization agreements;
- require the server to maintain histories of past changes to DIT
content and/or meta information;
- require the server to maintain synchronization state on a per
client basis; and/or
- are overly chatty.
Zeilenga LDAP Content Sync Operation [Page 2]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
Zeilenga LDAP Content Sync Operation [Page 3]
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
reload);
3) require pre-arranged synchronization agreements;
4) require the server to maintain synchronization state on a per
client basis;
5) require the server to maintain histories of past changes to DIT
content and/or meta information; and/or
6) are overly chatty.
The Sync operation provides eventual convergence of synchronized
content when possible and, when not, notification that content reload
The Sync Operation provides eventual convergence of synchronized
content when possible and, when not, notification that a full reload
is required.
The Sync operation does not require pre-arranged synchronization
The Sync Operation does not require pre-arranged synchronization
agreements.
The Sync operation does not require servers to maintain
synchronization state on a per user basis.
The Sync operation does not require servers to maintain any history of
past changes to the DIT or to meta information. While histories
(e.g., change logs, tombstones, DIT snapshots) may be used in the
implementation of the Sync operation, the operation may be implemented
using purely state-based approaches.
As the Sync operation does not require servers to maintain any
histories of past changes, it can be implemented in environments where
it is not feasible to maintain such histories. Histories, if
available, may be used by the server to reduce the number of messages
generated and reduce their size.
The Sync operation chattiness is reasonably bound.
The Sync Operation does not require servers to maintain nor to use any
history of past changes to the DIT or to meta information. However,
servers may maintain and use histories (e.g., change logs, tombstones,
DIT snapshots) to reduce the number of messages generated and to
reduce their size. As it is not always feasible to maintain and use
histories, the operation may be implemented using purely (current)
state-based approaches. The Sync Operation allows use of either the
state-based approach or the history-based approach in an operation by
operation basis to balance the size of history and the amount of
traffic. The Sync Operation also allows the combined use of the
state-based and the history-based approaches.
The Sync Operation does not require servers to maintain
synchronization state on a per client basis. However, servers may
maintain and use per client state information to reduce the number of
messages generated and the size of such messages.
A synchronization mechanism can be considered overly chatty when
synchronization traffic is not reasonably bounded. The Sync Operation
traffic is bounded by the size of updated (or new) entries and the
number of unchanged entries in the content. The operation is designed
to avoid full content exchanges even in the case that the history
information available to the server is insufficient to determine the
client's state. The operation is also designed to avoid transmission
of out-of-content history information, as its size is not bounded by
the content and it is not always feasible to transmit such history
information due to security reasons.
This document includes a number of non-normative appendices providing
additional information to server implementors.
1.2. Intended Usage
The Sync operation is intended to be used in applications requiring
eventual-convergent content synchronization. Upon completion of each
synchronization phase of the operation, all information to construct
an synchronized shadow copy of the content has been provided to the
client or the client has been notified that a complete content reload
is necessary. Excepting for transient inconsistencies due to
concurrent operation (or other) processing at the server, the shadow
copy is an accurate reflection of the content held by the server.
Each inconsistency is transient in that it will be corrected during
subsequent synchronization requests.
Possible uses include:
- White page service applications may use the Sync operation to
The Sync Operation is intended to be used in applications requiring
eventually-convergent content synchronization. Upon completion of
each synchronization stage of the operation, all information to
construct a synchronized client copy of the content has been provided
to the client or the client has been notified that a complete content
reload is necessary. Except for transient inconsistencies due to
concurrent operation (or other) processing at the server, the client
Zeilenga LDAP Content Sync Operation [Page 3]
Zeilenga LDAP Content Sync Operation [Page 4]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
maintain current shadow copy of a DIT fragment. For example, an
mail user agent which use the sync operation to maintain a local
copy of an enterprise address book.
copy is an accurate reflection of the content held by the server.
Transient inconsistencies will be resolved by subsequent
synchronization operations.
- Meta-information engines may use the Sync operation to maintain a
shadow copy of a DIT fragment.
Possible uses include:
- White page service applications may use the Sync Operation to
maintain current copy of a DIT fragment. For example, a mail user
agent which uses the sync operation to maintain a local copy of an
enterprise address book.
- Meta-information engines may use the Sync Operation to maintain a
copy of a DIT fragment.
- Caching proxy services may use the Sync operation to maintain a
- Caching proxy services may use the Sync Operation to maintain a
coherent content cache.
- Lightweight master-slave replication between heterogeneous
directory servers. For example, the Sync operation can be used by
directory servers. For example, the Sync Operation can be used by
a slave server to maintain a shadow copy of a DIT fragment.
Note: The International Telephone Union (ITU) has defined the X.500
Directory Synchronization Protocol [X.525] which may be used for
master-slave replication between LDAP servers. Other
experimental LDAP replication protocols exist. The Sync
operation should be viewed as complementary to these replication
protocols.
(Note: The International Telephone Union (ITU) has defined the
X.500 Directory [X.500] Information Shadowing Protocol (DISP)
[X.525] which may be used for master-slave replication between
directory servers. Other experimental LDAP replication protocols
also exist.)
This protocol is not intended to be used in applications requiring
transactional data consistency.
As this protocol transfers all visible values of entries upon change
instead of change deltas, this protocol is not appropriate for
bandwidth-challenged applications or deployments.
As this protocol transfers all visible values of entries belonging to
the content upon change instead of change deltas, this protocol is not
appropriate for bandwidth-challenged applications or deployments.
1.3. Overview
This section provides an overview of basis ways the Sync operation can
be used to maintain a synchronized shadow copy of a DIT fragment.
This section provides an overview of basic ways the Sync Operation can
be used to maintain a synchronized client copy of a DIT fragment.
- Polling for Changes: refreshOnly mode
- Listening for Changes: refreshAndPersist mode
......@@ -212,166 +268,216 @@ INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
1.3.1. Polling for Changes (refreshOnly)
To obtain its initial shadow copy, the client issues a Sync request: a
To obtain its initial client copy, the client issues a Sync request: a
search request with the Sync Request Control with mode set to
refreshOnly. The server, much like it would with a normal search
operation, returns (subject to access controls and other restrictions)
the content matching the search criteria (baseObject, scope, filter).
Additionally, with each entry returned, the server provides a Sync
State control indicating state add. This control contains the
Universally Unique Identifier (UUID) [UUID] of the entry. Unlike
Zeilenga LDAP Content Sync Operation [Page 4]
Zeilenga LDAP Content Sync Operation [Page 5]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
Distinguished Names (DNs), which may change over time, an entry's
UUIDs are stable. The initial content is followed by a
searchResultDone with a Sync Done control. The Sync Done control
the content matching the search criteria (baseObject, scope, filter,
attributes). Additionally, with each entry returned, the server
provides a Sync State Control indicating state add. This control
contains the Universally Unique Identifier (UUID) [UUID] of the entry
[EntryUUID]. Unlike the Distinguished Name (DN), which may change
over time, an entry's UUID is stable. The initial content is followed
by a SearchResultDone with a Sync Done Control. The Sync Done Control
provides a syncCookie. The syncCookie represents session state.
To poll for updates to the shadow copy, the client reissues the Sync
operation with the syncCookie previously returned. The server, much
To poll for updates to the client copy, the client reissues the Sync
Operation with the syncCookie previously returned. The server, much
as it would with a normal search operation, determines which content
would be returned as if the operation was a normal search operation.
However, using the syncCookie as an indicator of what content the
client was sent previously, the server sends copies of entries which
have changed with a Sync State control indicating state add. For each
unchanged entry, the server sends an empty entry (e.g., no attributes)
with a Sync State control indicating state present. The set of
updates is followed by a searchResultDone with a Sync Done control.
have changed with a Sync State Control indicating state add. For each
changed entry, all (modified or unmodified) attributes belonging to
the content are sent.
The server may perform either or both of the two distinct
synchronization phases which are distinguished by how to synchronize
entries deleted from the content: the present and the delete phases.
When the server uses a single phase for the refresh stage, each phase
is marked as ended by a SearchResultDone with a Sync Done Control. A
present phase is identified by a FALSE refreshDeletes value in the
Sync Done Control. A delete phase is identified by a TRUE
refreshDeletes value. The present phase may be followed by a delete
phase. The two phases are delimited by a refreshPresent Sync Info
Message having a FALSE refreshDone value. In the case that both the
phases are used, the present phase is used to bring the client copy up
to the state at which the subsequent delete phase can begin.
In the present phase, the server sends an empty entry (i.e., no
attributes) with a Sync State Control indicating state present for
each unchanged entry.
The delete phase may be used when the server can reliably determine
which entries in the prior client copy are no longer present in the
content and the number of such entries is less than or equal to the
number of unchanged entries. In the delete mode, the server sends an
empty entry with a Sync State Control indicating state delete for each
entry which is no longer in the content, instead of returning an empty
entry with state present for each present entry.
The server may send syncIdSet Sync Info Messages containing the set of
UUIDs of either unchanged present entries or deleted entries, instead
of sending multiple individual messages. If refreshDeletes of
syncIdSet is set to FALSE, the UUIDs of unchanged present entries are
If the server can reliably determine which entries in the prior shadow
copy are no longer present in the content and the number of such
entries is less than or equal to the number of unchanged entries, the
server may, instead of returning an empty entry with state present for
each present entry, send an empty entry with state delete for each
entry which is no longer in the content. Also, the Sync Done control
refreshDeletes is set to TRUE to indicate to the client that this
method was used. This field is FALSE otherwise.
The synchronized shadow copy of the DIT fragment is constructed by the
Zeilenga LDAP Content Sync Operation [Page 6]
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
contained in the syncUUIDs set; if refreshDeletes of syncIdSet is set
to TRUE, the UUIDs of the entries no longer present in the content are
contained in the syncUUIDs set. An optional cookie can be included in
the syncIdSet to represent the state of the content after
synchronizing the presence or the absence of the entries contained in
the syncUUIDs set.
The synchronized copy of the DIT fragment is constructed by the
client.
If refreshDeletes is FALSE, the new copy includes all changed entries
returned by the reissued Sync operation as well as all unchanged
entries identified as being present by the reissued Sync operation,
but whose content is provided by the previous Sync operation. The
unchanged entries not identified as being present are deleted from the
shadow content. They had been either deleted, moved, or otherwise
scoped-out from the content.
If refreshDeletes of syncDoneValue is FALSE, the new copy includes all
changed entries returned by the reissued Sync Operation as well as all
unchanged entries identified as being present by the reissued Sync
Operation, but whose content is provided by the previous Sync
Operation. The unchanged entries not identified as being present are
deleted from the client content. They had been either deleted, moved,
or otherwise scoped-out from the content.
If refreshDeletes is TRUE, the new copy includes all changed entries
returned by the reissued Sync operation as well as all other entries
of the previous copy except those which were identified as having been
deleted from the content.
If refreshDeletes of syncDoneValue is TRUE, the new copy includes all
changed entries returned by the reissued Sync Operation as well as all
other entries of the previous copy except for those which are
identified as having been deleted from the content.
The client can, at some later time, re-poll for changes to this
synchronized shadow copy.
synchronized client copy.
1.3.2. Listening for Changes (refreshAndPersist)
Polling for changes can be expensive in terms of server, client, and
network resources. The refreshAndPersist mode allows for active
updates of changed entries in the content.
By selecting the refreshAndPersist mode, the client requests the
server to send updates of entries that are changed after the initial
refresh content is determined. Instead of sending a SearchResultDone
Message as in polling, the server sends a Sync Info Message to the
client indicating that the refresh stage is complete and then enters
the persist stage. After receipt of this Sync Info Message, the
client will construct a synchronized copy as described in Section
1.3.1.
The server may then send change notifications as the result of the
original Sync search request which now remains persistent in the
server. For entries to be added to the returned content, the server
sends a SearchResultEntry (with attributes) with a Sync State Control
indicating state add. For entries to be deleted from the content, the
server sends a SearchResultEntry containing no attributes and a Sync
Zeilenga LDAP Content Sync Operation [Page 5]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
network resources. The refreshAndPersist mode allows for active
updates of changed entries in the content.
Zeilenga LDAP Content Sync Operation [Page 7]
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
By selecting the refreshAndPersist mode, the client requests the
server to send updates of entries that are changed after the the
initial refresh content is determined. Instead of sending a
searchResultDone message as described above, the server sends a Sync
Info message to the client indicating that refresh phase is complete
and then enters persist phase. After receipt of this Sync Info
message, the client will have a synchronized shadow copy as described
above.
The server may then send change notifications. For entries to be
added to the returned content, the server sends a searchResultEntry
(with attributes) with a Sync State control indicating state add. For
entries to be deleted from the content, the server sends a
searchResultEntry containing with no attributes and a Sync State
control indicating state delete. To modify entries in the return
content, the server sends a searchResultEntry (with attributes) with a
Sync State control indicating state modify. Upon modification of an
entry, all (modified or unmodified) attributes belonging to the
content are sent.
State Control indicating state delete. For entries to be modified in
the return content, the server sends a SearchResultEntry (with
attributes) with a Sync State Control indicating state modify. Upon
modification of an entry, all (modified or unmodified) attributes
belonging to the content are sent.
Note that renaming an entry of the DIT may cause an add state change
where the entry is renamed into the content, a delete state change
where the entry is renamed out of the content, and a modify state
change where the entry remains in the content. Also note that a
modification of an entry of the DIT may cause a add, delete, or modify
state change to the content.
modification of an entry of the DIT may cause an add, delete, or
modify state change to the content.
Upon receipt of a change notification, the client updates its copy of
the content.
If the server desires to update the syncCookie during the persist
stage, it may include the syncCookie any Sync State control or Sync
Info message returned.
stage, it may include the syncCookie in any Sync State Control or Sync
Info Message returned.
The operation persists until canceled [CANCEL] by the client or
terminated by the server. A Sync Done control may be attached to
searchResultDone message to provide a new syncCookie.
terminated by the server. A Sync Done Control shall be attached to
SearchResultDone Message to provide a new syncCookie.
1.4. Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14 [RFC2119].
Protocol elements are described using ASN.1 [X.680] with implicit
tags. The term "BER-encoded" means the element is to be encoded using
the Basic Encoding Rules [X.690] under the restrictions detailed in
Section 5.1 of [RFC2251].
2. Elements of the Sync Operation
The Sync Operation is defined as an extension to the LDAP Search
Operation [RFC2251] where the directory user agent (DUA or client)
submits a SearchRequest message with a Sync Request control and the
submits a SearchRequest Message with a Sync Request Control and the
directory system agent (DSA or server) responses with zero or more
SearchResultEntry Messages, each with a Sync State Control; zero or
more SearchResultReference Messages, each with a Sync State Control;
zero or more Sync Info Intermediate Response Messages; and a
SearchResultDone Message with a Sync Done Control.
Zeilenga LDAP Content Sync Operation [Page 6]
INTERNET-DRAFT draft-zeilenga-ldup-sync-02 5 May 2003
Zeilenga LDAP Content Sync Operation [Page 8]
INTERNET-DRAFT draft-zeilenga-ldup-sync-05 3 February 2004
SearchResultEntry messages, each with a Sync State control; zero or
more SearchResultReference messages, each with a Sync State control;
zero or more Sync Intermediate Response messages; and a
searchResultDone message with a Sync Done control.
To allow clients to discover support for this operation, servers
implementing this operation SHOULD publish the IANA-ASSIGNED-OID.1 as
a value of supportedControl root DSE attribute.
implementing this operation SHOULD publish the
1.3.6.1.4.1.4203.1.9.1.1 as a value of 'supportedControl' attribute
[RFC2252] of the root DSA-specific entry (DSE). A server MAY choose
to advertise this extension only when the client is authorized to use
it.
2.1 Common ASN.1 elements
2.1 Common ASN.1 Elements
2.1.1 syncUUID
The syncUUID is a notational convenience to indicate that, while the
syncUUID type is encoded as an OCTET STRING, its value is restricted
to the string representation of an Universally Unique Identifier