Commit 20cc2dd6 authored by Quanah Gibson-Mount's avatar Quanah Gibson-Mount
Browse files

ITS#5746

work in progress. Expand introduction, move slurpd-replacement lower.
parent 24807705
......@@ -21,6 +21,8 @@ OpenLDAP 2.4.13 Engineering
Fixed slapo-rwm reusing freed filter (ITS#5732)
Added slapo-translucent try local bind when remote fails (ITS#5656)
Added slapo-translucent support for PasswordModify exop (ITS#5656)
Documentation
admin24 added olcLimits to example (ITS#5746)
OpenLDAP 2.4.12 Release (2008/10/12)
Fixed libldap ldap_utf8_strchar arguments (ITS#5720)
......
......@@ -8,293 +8,50 @@ Replicated directories are a fundamental requirement for delivering a
resilient enterprise deployment.
{{PRD:OpenLDAP}} has various configuration options for creating a replicated
directory. The following sections will discuss these.
H2: Push Based
H3: Replacing Slurpd
{{Slurpd}} replication has been deprecated in favor of Syncrepl replication and
has been completely removed from OpenLDAP 2.4.
{{Why was it replaced?}}
The {{slurpd}} daemon was the original replication mechanism inherited from
UMich's LDAP and operates in push mode: the master pushes changes to the
slaves. It has been replaced for many reasons, in brief:
* It is not reliable
* It is extremely sensitive to the ordering of records in the replog
* It can easily go out of sync, at which point manual intervention is
required to resync the slave database with the master directory
* It isn't very tolerant of unavailable servers. If a slave goes down
for a long time, the replog may grow to a size that's too large for
slurpd to process
{{What was it replaced with?}}
Syncrepl
{{Why is Syncrepl better?}}
* Syncrepl is self-synchronizing; you can start with a database in any
state from totally empty to fully synced and it will automatically do
the right thing to achieve and maintain synchronization
* Syncrepl can operate in either direction
* Data updates can be minimal or maximal
{{How do I implement a pushed based replication system using Syncrepl?}}
The easiest way is to point an LDAP backend ({{SECT: Backends}} and {{slapd-ldap(8)}})
to your slave directory and setup Syncrepl to point to your Master database.
If you imagine Syncrepl pulling down changes from the Master server, and then
pushing those changes out to your slave servers via {{slapd-ldap(8)}}. This is
called Syncrepl Proxy Mode. You can also use Syncrepl Multi-proxy mode:
!import "push-based-complete.png"; align="center"; title="Syncrepl Proxy Mode"
FT[align="Center"] Figure X.Y: Replacing slurpd
The following example is for a self-contained push-based replication solution:
> #######################################################################
> # Standard OpenLDAP Master/Provider
> #######################################################################
>
> include /usr/local/etc/openldap/schema/core.schema
> include /usr/local/etc/openldap/schema/cosine.schema
> include /usr/local/etc/openldap/schema/nis.schema
> include /usr/local/etc/openldap/schema/inetorgperson.schema
>
> include /usr/local/etc/openldap/slapd.acl
>
> modulepath /usr/local/libexec/openldap
> moduleload back_hdb.la
> moduleload syncprov.la
> moduleload back_monitor.la
> moduleload back_ldap.la
>
> pidfile /usr/local/var/slapd.pid
> argsfile /usr/local/var/slapd.args
>
> loglevel sync stats
>
> database hdb
> suffix "dc=suretecsystems,dc=com"
> directory /usr/local/var/openldap-data
>
> checkpoint 1024 5
> cachesize 10000
> idlcachesize 10000
>
> index objectClass eq
> # rest of indexes
> index default sub
>
> rootdn "cn=admin,dc=suretecsystems,dc=com"
> rootpw testing
>
> # syncprov specific indexing
> index entryCSN eq
> index entryUUID eq
>
> # syncrepl Provider for primary db
> overlay syncprov
> syncprov-checkpoint 1000 60
>
> # Let the replica DN have limitless searches
> limits dn.exact="cn=replicator,dc=suretecsystems,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
>
> database monitor
>
> database config
> rootpw testing
>
> ##############################################################################
> # Consumer Proxy that pulls in data via Syncrepl and pushes out via slapd-ldap
> ##############################################################################
>
> database ldap
> # ignore conflicts with other databases, as we need to push out to same suffix
> hidden on
> suffix "dc=suretecsystems,dc=com"
> rootdn "cn=slapd-ldap"
> uri ldap://localhost:9012/
>
> lastmod on
>
> # We don't need any access to this DSA
> restrict all
>
> acl-bind bindmethod=simple
> binddn="cn=replicator,dc=suretecsystems,dc=com"
> credentials=testing
>
> syncrepl rid=001
> provider=ldap://localhost:9011/
> binddn="cn=replicator,dc=suretecsystems,dc=com"
> bindmethod=simple
> credentials=testing
> searchbase="dc=suretecsystems,dc=com"
> type=refreshAndPersist
> retry="5 5 300 5"
>
> overlay syncprov
A replica configuration for this type of setup could be:
> #######################################################################
> # Standard OpenLDAP Slave without Syncrepl
> #######################################################################
>
> include /usr/local/etc/openldap/schema/core.schema
> include /usr/local/etc/openldap/schema/cosine.schema
> include /usr/local/etc/openldap/schema/nis.schema
> include /usr/local/etc/openldap/schema/inetorgperson.schema
>
> include /usr/local/etc/openldap/slapd.acl
>
> modulepath /usr/local/libexec/openldap
> moduleload back_hdb.la
> moduleload syncprov.la
> moduleload back_monitor.la
> moduleload back_ldap.la
>
> pidfile /usr/local/var/slapd.pid
> argsfile /usr/local/var/slapd.args
>
> loglevel sync stats
>
> database hdb
> suffix "dc=suretecsystems,dc=com"
> directory /usr/local/var/openldap-slave/data
>
> checkpoint 1024 5
> cachesize 10000
> idlcachesize 10000
>
> index objectClass eq
> # rest of indexes
> index default sub
>
> rootdn "cn=admin,dc=suretecsystems,dc=com"
> rootpw testing
>
> # Let the replica DN have limitless searches
> limits dn.exact="cn=replicator,dc=suretecsystems,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
>
> updatedn "cn=replicator,dc=suretecsystems,dc=com"
>
> # Refer updates to the master
> updateref ldap://localhost:9011
>
> database monitor
>
> database config
> rootpw testing
You can see we use the {{updatedn}} directive here and example ACLs ({{F:usr/local/etc/openldap/slapd.acl}}) for this could be:
> # Give the replica DN unlimited read access. This ACL may need to be
> # merged with other ACL statements.
>
> access to *
> by dn.base="cn=replicator,dc=suretecsystems,dc=com" write
> by * break
>
> access to dn.base=""
> by * read
>
> access to dn.base="cn=Subschema"
> by * read
>
> access to dn.subtree="cn=Monitor"
> by dn.exact="uid=admin,dc=suretecsystems,dc=com" write
> by users read
> by * none
>
> access to *
> by self write
> by * read
In order to support more replicas, just add more {{database ldap}} sections and
increment the {{syncrepl rid}} number accordingly.
Note: You must populate the Master and Slave directories with the same data,
unlike when using normal Syncrepl
If you do not have access to modify the master directory configuration you can
configure a standalone ldap proxy, which might look like:
!import "push-based-standalone.png"; align="center"; title="Syncrepl Standalone Proxy Mode"
FT[align="Center"] Figure X.Y: Replacing slurpd with a standalone version
The following configuration is an example of a standalone LDAP Proxy:
> include /usr/local/etc/openldap/schema/core.schema
> include /usr/local/etc/openldap/schema/cosine.schema
> include /usr/local/etc/openldap/schema/nis.schema
> include /usr/local/etc/openldap/schema/inetorgperson.schema
>
> include /usr/local/etc/openldap/slapd.acl
>
> modulepath /usr/local/libexec/openldap
> moduleload syncprov.la
> moduleload back_ldap.la
>
> ##############################################################################
> # Consumer Proxy that pulls in data via Syncrepl and pushes out via slapd-ldap
> ##############################################################################
>
> database ldap
> # ignore conflicts with other databases, as we need to push out to same suffix
> hidden on
> suffix "dc=suretecsystems,dc=com"
> rootdn "cn=slapd-ldap"
> uri ldap://localhost:9012/
>
> lastmod on
>
> # We don't need any access to this DSA
> restrict all
>
> acl-bind bindmethod=simple
> binddn="cn=replicator,dc=suretecsystems,dc=com"
> credentials=testing
>
> syncrepl rid=001
> provider=ldap://localhost:9011/
> binddn="cn=replicator,dc=suretecsystems,dc=com"
> bindmethod=simple
> credentials=testing
> searchbase="dc=suretecsystems,dc=com"
> type=refreshAndPersist
> retry="5 5 300 5"
>
> overlay syncprov
As you can see, you can let your imagination go wild using Syncrepl and
{{slapd-ldap(8)}} tailoring your replication to fit your specific network
topology.
H2: Pull Based
directory. In previous releases, replication was discussed in terms of
a {{master}} server and some number of {{slave}} servers. A master
accepted directory updates from other clients, and a slave only
accepted updates from a (single) master. The replication structure
was rigidly defined and any particular database could only fulfill
a single role, either master or slave.
As OpenLDAP now supports a wide variety of replication topologies, these
terms have been deprecated in favor of {{provider}} and
{{consumer}}: A provider replicates directory updates to consumers;
consumers receive replication updates from providers. Unlike the
rigidly defined master/slave relationships, provider/consumer roles
are quite fluid: replication updates received in a consumer can be
further propagated by that consumer to other servers, so a consumer
can also act simultaneously as a provider. Also, a consumer need not
be an actual LDAP server; it may be just an LDAP client.
The following sections will describe the replication technology and
discuss the various replication options that are available.
H2: Replication Technology
H3: LDAP Sync Replication
The {{TERM:LDAP Sync}} Replication engine, {{TERM:syncrepl}} for
short, is a consumer-side replication engine that enables the
consumer {{TERM:LDAP}} server to maintain a shadow copy of a
{{TERM:DIT}} fragment. A syncrepl engine resides at the consumer-side
as one of the {{slapd}}(8) threads. It creates and maintains a
{{TERM:DIT}} fragment. A syncrepl engine resides at the consumer
and executes as one of the {{slapd}}(8) threads. It creates and maintains a
consumer replica by connecting to the replication provider to perform
the initial DIT content load followed either by periodic content
polling or by timely updates upon content changes.
Syncrepl uses the LDAP Content Synchronization (or LDAP Sync for
short) protocol as the replica synchronization protocol. It provides
Syncrepl uses the LDAP Content Synchronization protocol (or LDAP Sync for
short) as the replica synchronization protocol. LDAP Sync provides
a stateful replication which supports both pull-based and push-based
synchronization and does not mandate the use of a history store.
In pull-based replication the consumer periodically
polls the provider for updates. In push-based replication the consumer
listens for updates that are sent by the provider in realtime. Since the
protocol does not require a history store, the provider does not need to
maintain any log of updates it has received. (Note
that the syncrepl engine is extensible and additional replication
protocols may be supported in the future.)
Syncrepl keeps track of the status of the replication content by
maintaining and exchanging synchronization cookies. Because the
......@@ -331,7 +88,7 @@ privileges for the DIT fragment to be replicated. The consumer
server can stop the replication also without the need for provider-side
changes and restart.
Syncrepl supports both partial and sparse replications. The shadow
Syncrepl supports partial, sparse, and fractional replications. The shadow
DIT fragment is defined by a general search criteria consisting of
base, scope, filter, and attribute list. The replica content is
also subject to the access privileges of the bind identity of the
......@@ -346,56 +103,60 @@ of controls and other protocol elements which extend the LDAP search
operation. This section introduces the LDAP Content Sync protocol
only briefly. For more information, refer to {{REF:RFC4533}}.
The LDAP Sync protocol supports both polling and listening for
changes by defining two respective synchronization operations:
The LDAP Sync protocol supports both polling and listening for changes
by defining two respective synchronization operations:
{{refreshOnly}} and {{refreshAndPersist}}. Polling is implemented
by the {{refreshOnly}} operation. The client copy is synchronized
to the server copy at the time of polling. The server finishes the
by the {{refreshOnly}} operation. The consumer
polls the provider using an LDAP Search request with an LDAP Sync
control attached. The consumer copy is synchronized
to the provider copy at the time of polling using the information
returned in the search. The provider finishes the
search operation by returning {{SearchResultDone}} at the end of
the search operation as in the normal search. The listening is
implemented by the {{refreshAndPersist}} operation. Instead of
the search operation as in the normal search. Listening is
implemented by the {{refreshAndPersist}} operation. As the name
implies, it begins with a search, like refreshOnly. Instead of
finishing the search after returning all entries currently matching
the search criteria, the synchronization search remains persistent
in the server. Subsequent updates to the synchronization content
in the server cause additional entry updates to be sent to the
client.
in the provider. Subsequent updates to the synchronization content
in the provider cause additional entry updates to be sent to the
consumer.
The {{refreshOnly}} operation and the refresh stage of the
{{refreshAndPersist}} operation can be performed with a present
phase or a delete phase.
In the present phase, the server sends the client the entries updated
within the search scope since the last synchronization. The server
sends all requested attributes, be it changed or not, of the updated
In the present phase, the provider sends the consumer the entries updated
within the search scope since the last synchronization. The provider
sends all requested attributes, be they changed or not, of the updated
entries. For each unchanged entry which remains in the scope, the
server sends a present message consisting only of the name of the
provider sends a present message consisting only of the name of the
entry and the synchronization control representing state present.
The present message does not contain any attributes of the entry.
After the client receives all update and present entries, it can
reliably determine the new client copy by adding the entries added
to the server, by replacing the entries modified at the server, and
by deleting entries in the client copy which have not been updated
nor specified as being present at the server.
After the consumer receives all update and present entries, it can
reliably determine the new consumer copy by adding the entries added
to the provider, by replacing the entries modified at the provider, and
by deleting entries in the consumer copy which have not been updated
nor specified as being present at the provider.
The transmission of the updated entries in the delete phase is the
same as in the present phase. The server sends all the requested
same as in the present phase. The provider sends all the requested
attributes of the entries updated within the search scope since the
last synchronization to the client. In the delete phase, however,
the server sends a delete message for each entry deleted from the
last synchronization to the consumer. In the delete phase, however,
the provider sends a delete message for each entry deleted from the
search scope, instead of sending present messages. The delete
message consists only of the name of the entry and the synchronization
control representing state delete. The new client copy can be
control representing state delete. The new consumer copy can be
determined by adding, modifying, and removing entries according to
the synchronization control attached to the {{SearchResultEntry}}
message.
In the case that the LDAP Sync server maintains a history store and
can determine which entries are scoped out of the client copy since
the last synchronization time, the server can use the delete phase.
If the server does not maintain any history store, cannot determine
In the case that the LDAP Sync provider maintains a history store and
can determine which entries are scoped out of the consumer copy since
the last synchronization time, the provider can use the delete phase.
If the provider does not maintain any history store, cannot determine
the scoped-out entries from the history store, or the history store
does not cover the outdated synchronization state of the client,
the server should use the present phase. The use of the present
does not cover the outdated synchronization state of the consumer,
the provider should use the present phase. The use of the present
phase is much more efficient than a full content reload in terms
of the synchronization traffic. To reduce the synchronization
traffic further, the LDAP Sync protocol also provides several
......@@ -403,22 +164,20 @@ optimizations such as the transmission of the normalized {{EX:entryUUID}}s
and the transmission of multiple {{EX:entryUUIDs}} in a single
{{syncIdSet}} message.
At the end of the {{refreshOnly}} synchronization, the server sends
a synchronization cookie to the client as a state indicator of the
client copy after the synchronization is completed. The client
At the end of the {{refreshOnly}} synchronization, the provider sends
a synchronization cookie to the consumer as a state indicator of the
consumer copy after the synchronization is completed. The consumer
will present the received cookie when it requests the next incremental
synchronization to the server.
synchronization to the provider.
When {{refreshAndPersist}} synchronization is used, the server sends
When {{refreshAndPersist}} synchronization is used, the provider sends
a synchronization cookie at the end of the refresh stage by sending
a Sync Info message with TRUE refreshDone. It also sends a
a Sync Info message with refreshDone=TRUE. It also sends a
synchronization cookie by attaching it to {{SearchResultEntry}}
generated in the persist stage of the synchronization search. During
the persist stage, the server can also send a Sync Info message
containing the synchronization cookie at any time the server wants
to update the client-side state indicator. The server also updates
a synchronization indicator of the client at the end of the persist
stage.
messages generated in the persist stage of the synchronization search. During
the persist stage, the provider can also send a Sync Info message
containing the synchronization cookie at any time the provider wants
to update the consumer-side state indicator.
In the LDAP Sync protocol, entries are uniquely identified by the
{{EX:entryUUID}} attribute value. It can function as a reliable
......@@ -428,7 +187,6 @@ reliable identifier. The {{EX:entryUUID}} is attached to each
{{SearchResultEntry}} or {{SearchResultReference}} as a part of the
synchronization control.
H4: Syncrepl Details
The syncrepl engine utilizes both the {{refreshOnly}} and the
......@@ -444,10 +202,10 @@ provider.
The syncrepl engine utilizes both the present phase and the delete
phase of the refresh synchronization. It is possible to configure
a per-scope session log in the provider server which stores the
a session log in the provider which stores the
{{EX:entryUUID}}s of a finite number of entries deleted from a
replication content. Multiple replicas of single provider content
share the same per-scope session log. The syncrepl engine uses the
database. Multiple replicas share the same session log. The syncrepl
engine uses the
delete phase if the session log is present and the state of the
consumer server is recent enough that no session log entries are
truncated after the last synchronization of the client. The syncrepl
......@@ -538,20 +296,33 @@ this change without the use of the session log.
For configuration, please see the {{SECT:Syncrepl}} section.
H2: Deployment Alternatives
While the LDAP Sync specification only defines a narrow scope for replication,
the OpenLDAP implementation is extremely flexible and supports a variety of
operating modes to handle other scenarios not explicitly addressed in the spec.
H3: Delta-syncrepl replication
* Disadvantages of Syncrepl replication:
* Disadvantages of LDAP Sync replication:
OpenLDAP's syncrepl replication is an object-based replication mechanism.
LDAP Sync replication is an object-based replication mechanism.
When any attribute value in a replicated object is changed on the provider,
each consumer fetches and processes the complete changed object {{B:both changed and unchanged attribute values}} during replication. This works well, but has drawbacks in some situations.
each consumer fetches and processes the complete changed object
{{B:both changed and unchanged attribute values}} during replication.
One advantage of this approach is that when multiple changes occur to
a single object, the precise sequence of those changes need not be preserved;
only the final state of the entry is significant. But this approach
may have drawbacks when the usage pattern involves single changes to
multiple objects.
For example, suppose you have a database consisting of 100,000 objects of 1 KB
each. Further, suppose you routinely run a batch job to change the value of
a single two-byte attribute value that appears in each of the 100,000 objects
on the master. Not counting LDAP and TCP/IP protocol overhead, each time you
run this job each consumer will transfer and process {{B:1 GB}} of data to process
{{B:200KB of changes!}}
run this job each consumer will transfer and process {{B:1 GB}} of data to
process {{B:200KB of changes!}}
99.98% of the data that is transmitted and processed in a case like this will
be redundant, since it represents values that did not change. This is a waste
......@@ -564,45 +335,55 @@ demonstrate a very real problem that is encountered in some LDAP deployments.
Delta-syncrepl, a changelog-based variant of syncrepl, is designed to address
situations like the one described above. Delta-syncrepl works by maintaining a
changelog of a selectable depth on the provider. The replication consumer on
each consumer checks the changelog for the changes it needs and, as long as
the changelog contains the needed changes, the delta-syncrepl consumer fetches
them from the changelog and applies them to its database. If, however, a replica
changelog of a selectable depth on the provider. The replication consumer
checks the changelog for the changes it needs and, as long as
the changelog contains the needed changes, the consumer fetches the changes
from the changelog and applies them to its database. If, however, a replica
is too far out of sync (or completely empty), conventional syncrepl is used to
bring it up to date and replication then switches to the delta-syncrepl mode.
bring it up to date and replication then switches back to the delta-syncrepl
mode.
For configuration, please see the {{SECT:Delta-syncrepl}} section.
H2: Mixture of both Pull and Push based
H3: N-Way Multi-Master replication
Multi-Master replication is a replication technique using Syncrepl to replicate
data to multiple Master Directory servers.
data to multiple provider ("Master") Directory servers.
* Advantages of Multi-Master replication:
H4: Valid Arguments for Multi-Master replication
- If any master fails, other masters will continue to accept updates
- Avoids a single point of failure
- Masters can be located in several physical sites i.e. distributed across the
network/globe.
- Good for Automatic failover/High Availability
* If any provider fails, other providers will continue to accept updates
* Avoids a single point of failure
* Providers can be located in several physical sites i.e. distributed across
the network/globe.
* Good for Automatic failover/High Availability
* Disadvantages of Multi-Master replication:
H4: Invalid Arguments for Multi-Master replication
- It has {{B:NOTHING}} to do with load balancing
- {{URL:http://www.openldap.org/faq/data/cache/1240.html}}
- If connectivity with a master is lost because of a network partition, then
(These are often claimed to be advantages of Multi-Master replication but
those claims are false):
* It has {{B:NOTHING}} to do with load balancing
* Providers {{B:must}} propagate writes to {{B:all}} the other servers, which
means the network traffic and write load spreads across all
of the servers the same as for single-master.
* Server utilization and performance are at best identical for
Multi-Master and Single-Master replication; at worst Single-Master is
superior because indexing can be tuned differently to optimize for the
different usage patterns between the provider and the consumers.
H4: Arguments against Multi-Master replication
* Breaks the data consistency guarantees of the directory model
* {{URL:http://www.openldap.org/faq/data/cache/1240.html}}
* If connectivity with a provider is lost because of a network partition, then
"automatic failover" can just compound the problem
- Typically, a particular machine cannot distinguish between losing contact
* Typically, a particular machine cannot distinguish between losing contact
with a peer because that peer crashed, or because the network link has failed
- If a network is partitioned and multiple clients start writing to each of the
* If a network is partitioned and multiple clients start writing to each of the
"masters" then reconciliation will be a pain; it may be best to simply deny
writes to the clients that are partitioned from the single master
- Masters {{B:must}} propagate writes to {{B:all}} the other servers, which
means the network traffic and write load is constant and spreads across all
of the servers
writes to the clients that are partitioned from the single provider
For configuration, please see the {{SECT:N-Way Multi-Master}} section below
......@@ -611,48 +392,95 @@ H3: MirrorMode replication
MirrorMode is a hybrid configuration that provides all of the consistency
guarantees of single-master replication, while also providing the high
availability of multi-master. In MirrorMode two masters are set up to
availability of multi-master. In MirrorMode two providers are set up to
replicate from each other (as a multi-master configuration), but an
external frontend is employed to direct all writes to only one of
the two servers. The second master will only be used for writes if
the first master crashes, at which point the frontend will switch to
directing all writes to the second master. When a crashed master is
the two servers. The second provider will only be used for writes if
the first provider crashes, at which point the frontend will switch to
directing all writes to the second provider. When a crashed provider is
repaired and restarted it will automatically catch up to any changes
on the running master and resync.
on the running provider and resync.
H4: Arguments for MirrorMode
* Provides a high-availability (HA) solution for directory writes (replicas handle reads)
* As long as one Master is operational, writes can safely be accepted
* Master nodes replicate from each other, so they are always up to date and
* As long as one provider is operational, writes can safely be accepted
* Provider nodes replicate from each other, so they are always up to date and
can be ready to take over (hot standby)
* Syncrepl also allows the master nodes to re-synchronize after any downtime
* Syncrepl also allows the provider nodes to re-synchronize after any downtime
* Delta-Syncrepl can be used
H4: Arguments against MirrorMode
* MirrorMode is not what is termed as a Multi-Master solution. This is because
writes have to go to one of the mirror nodes at a time
* MirrorMode can be termed as Active-Active Hot-Standby, therefor an external
server (slapd in proxy mode) or device (hardware load balancer) to manage which
master is currently active
* While syncrepl can recover from a completely empty database, slapadd is much