[389-users] bulk initialization with MMR

[Luke Bigum]

Hi Gary,

I used to work for a university that does something similar to what you are trying to do. I'll explain their setup and it might give you a few ideas. They have a custom user management database that's the authoritative source of computer account information, a series of FDS servers are used for identification and authentication. A Perl script is used to turn the database contents into LDIF format as it would be used to populate an empty database (like one of your ldif2db batch extracts). They then take a dump of the LDAP directory into LDIF format and compare the database LDIF to directory LDIF and come up with a delta LDIF file. This delta LDIF is then run on the directory server to bring it in line with the database contents.

This update process runs every couple minutes, so the delta never really gets that big and password changes / new users only take a few minutes to propagate around the university. They would never need to batch import the entire database contents unless there was a catastrophe.

So, for your scenario, you might consider scrapping the nightly bulk initialisations, turn your servers into MMR and look at doing more frequent updates with delta files to provide faster synchronisation between your data sources.

If you actually need to do real real-time updates, you can do that with the same setup above, you just need to fire off a specific LDAP update to your load balanced LDAP from Peoplesoft.

Luke Bigum
Systems Administrator
 (p) 1300 661 668
 (f)  1300 661 540
(e)  lbigum at iseek.com.au
http://www.iseek.com.au
Level 1, 100 Ipswich Road Woolloongabba QLD 4102



This e-mail and any files transmitted with it may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorised to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message.


-----Original Message-----
From: fedora-directory-users-bounces at redhat.com [mailto:fedora-directory-users-bounces at redhat.com] On Behalf Of Rich Megginson
Sent: Wednesday 1 July 2009 1:21 AM
To: General discussion list for the 389 Directory server project.
Subject: Re: [389-users] bulk initialization with MMR

Gary Windham wrote:
> We have a setup where we are running 2 servers behind a load balancer
> (for HA purposes), where each of these servers is bulk-initialized
> daily (via ldif2db.pl) with a large set of data fed to us via batch
> extracts from various administrative systems.  Up till now, there has
> been no need to configure replication between these 2 servers, as all
> of the data is read-only.  However, we now have a requirement to
> update some of the directory data in a "real-time" fashion (e.g., when
> particular events fire in our PeopleSoft system we want to update the
> directory)--hence, the need for MMR.  The batch extracts will still be
> our "checkpoints", so we will want to load them in once-per-day, as we
> do now.
How does the data get from peoplesoft to the directory server?
>
> So, the question is: what would be the "recommended" approach for a
> scenario like this?  How do we (can we?) make MMR coexist peacefully
> with frequent bulk initializations?
In general, it's not a good idea to do a bulk load daily.
>
> TIA,
> --Gary
>
> --
> Gary Windham
> Senior Enterprise Systems Architect
> The University of Arizona, UITS
> +1 520 626 5981
>
> --
> 389 users mailing list
> 389-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users