[Fedora-directory-users] surgery on existing directory

[Graham Seaman]

Hi,

I have an existing populated directory supporting a live application. 
The next development version will have some fairly large scale changes - 
changes to schema, objectClasses, attribute names and attribute values - 
but I can't lose the actual data we already have.

The approach I've been trying is:

1. Use db2ldif to dump the groups and users (the only bit of the data 
which is 'mine') from the live directory on the live system:

/usr/lib/dirsrv/slapd-flame/db2ldif -U -n userRoot -a 
/opt/backups/original.ldif -s "dc=lse,dc=ac,dc=uk" -s "ou=My Groups"
-s 
"ou=My Users"

2. Edit the ldif file with the changes I need

3. Load the ldif file  into a  new fedora directory  on my development 
system with ldif2db.pl:

/usr/lib/dirsrv/slapd-dam/ldif2db.pl -D "cn=directory manager" -w MYPASS 
-n userRoot -s  "dc=lse,dc=ac,dc=uk"  -s "ou=New Groups" -s
"ou=New 
Users" -i /opt/backups/new.ldif

ldif2db.pl terminates almost immediately, clearly without having read 
most of the file. The fedora log shows:

[14/Nov/2008:11:35:54 +0000] conn=2 op=1 ADD 
dn="cn=import_2008_11_14_11_35_55, cn=import, cn=tasks, cn=config"
[14/Nov/2008:11:35:54 +0000] conn=2 op=1 RESULT err=0 tag=105 nentries=0 
etime=0

If I repeat the operation I get 'operation error';  and if I try to 
access the directory, it appears to be completely empty.

So, two questions:

- is this a reasonable way to go about this task, or are there other 
tools I should use?
- any suggestions for debugging?

Thanks
Graham

[Rich Megginson]

Graham Seaman wrote:
>  Hi,
>
>  I have an existing populated directory supporting a live application. 
>  The next development version will have some fairly large scale changes 
>  - changes to schema, objectClasses, attribute names and attribute 
>  values - but I can't lose the actual data we already have.
>
>  The approach I've been trying is:
>
>  1. Use db2ldif to dump the groups and users (the only bit of the data 
>  which is 'mine') from the live directory on the live system:
>
>  /usr/lib/dirsrv/slapd-flame/db2ldif -U -n userRoot -a 
>  /opt/backups/original.ldif -s "dc=lse,dc=ac,dc=uk" -s "ou=My Groups"
>
>  -s "ou=My Users" 
That's not really the purpose of db2ldif - it really
wants to operate on 
the entire contents of the database.  If you need to edit selective 
parts of the tree, you'll have to use one of the following approaches:
Use db2ldif to get everything, but only modify the parts you want
Use ldapsearch to selectively get what you want - then, either use 
ldapdelete to remove entire entries and ldapmodify -a to add them back, 
or if you can just modify the entries in place, use ldif change 
statements (changetype: modify) and use ldapmodify (no -a)
>  2. Edit the ldif file with the changes I need
>
>  3. Load the ldif file  into a  new fedora directory  on my development 
>  system with ldif2db.pl:
>
>  /usr/lib/dirsrv/slapd-dam/ldif2db.pl -D "cn=directory manager" -w 
>  MYPASS -n userRoot -s  "dc=lse,dc=ac,dc=uk"  -s "ou=New Groups" -s 
>  "ou=New Users" -i /opt/backups/new.ldif
>
>  ldif2db.pl terminates almost immediately, clearly without having read 
>  most of the file. The fedora log shows:
>
>  [14/Nov/2008:11:35:54 +0000] conn=2 op=1 ADD 
>  dn="cn=import_2008_11_14_11_35_55, cn=import, cn=tasks, cn=config"
>  [14/Nov/2008:11:35:54 +0000] conn=2 op=1 RESULT err=0 tag=105 
>  nentries=0 etime=0 
This is because ldif2db.pl just invokes an internal task using a
task 
entry.  If you want to monitor the progress of the import, you'll have 
to look at the errors log, or use ldapsearch to query the entry that 
ldif2db.pl spits out (cn=import_2008_11_14_11_35_55, cn=import, 
cn=tasks, cn=config)
>  If I repeat the operation I get 'operation error';  and if I try to 
>  access the directory, it appears to be completely empty. 
Probably what happened is
that you attempted to import from an LDIF file 
that did not contain the parent entries - the LDIF only contained your 
users and groups.  Import (ldif2db) is a _destructive_ operation - it 
will completely wipe out the contents of your database before adding the 
new entries.  In order to add an entry in LDAP, the parent entry must 
exist.   This means that if you want to import an LDIF file, and 
dc=lse,dc=ac,dc=uk is your base suffix, the LDIF file must contain the entry
dn: dc=lse,dc=ac,dc=uk
...

and any other parent entries of the entries you want to import.
>  So, two questions:
>
>  - is this a reasonable way to go about this task, or are there other 
>  tools I should use?
>  - any suggestions for debugging?
>
>  Thanks
>  Graham
>
>
>
>
>
>  -- 
>  Fedora-directory-users mailing list
>  Fedora-directory-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users

[Graham Seaman]

Dumping the entire thing, editing, and restoring worked fine. Thank you!

Graham

Rich Megginson wrote:
>  Graham Seaman wrote:
> > Hi,
> >
> > I have an existing populated directory supporting a live application. 
> > The next development version will have some fairly large scale 
> > changes - changes to schema, objectClasses, attribute names and 
> > attribute values - but I can't lose the actual data we already have.
> >
> > The approach I've been trying is:
> >
> > 1. Use db2ldif to dump the groups and users (the only bit of the data 
> > which is 'mine') from the live directory on the live system:
> >
> > /usr/lib/dirsrv/slapd-flame/db2ldif -U -n userRoot -a 
> > /opt/backups/original.ldif -s "dc=lse,dc=ac,dc=uk" -s "ou=My
> Groups" 
> > -s "ou=My Users"
>  That's not really the purpose of db2ldif - it really wants to operate 
>  on the entire contents of the database.  If you need to edit selective 
>  parts of the tree, you'll have to use one of the following approaches:
>  Use db2ldif to get everything, but only modify the parts you want
>  Use ldapsearch to selectively get what you want - then, either use 
>  ldapdelete to remove entire entries and ldapmodify -a to add them 
>  back, or if you can just modify the entries in place, use ldif change 
>  statements (changetype: modify) and use ldapmodify (no -a)
> >
> > 2. Edit the ldif file with the changes I need
> >
> > 3. Load the ldif file  into a  new fedora directory  on my 
> > development system with ldif2db.pl:
> >
> > /usr/lib/dirsrv/slapd-dam/ldif2db.pl -D "cn=directory manager" -w 
> > MYPASS -n userRoot -s  "dc=lse,dc=ac,dc=uk"  -s "ou=New Groups"
> -s 
> > "ou=New Users" -i /opt/backups/new.ldif
> >
> > ldif2db.pl terminates almost immediately, clearly without having read 
> > most of the file. The fedora log shows:
> >
> > [14/Nov/2008:11:35:54 +0000] conn=2 op=1 ADD 
> > dn="cn=import_2008_11_14_11_35_55, cn=import, cn=tasks, cn=config"
> > [14/Nov/2008:11:35:54 +0000] conn=2 op=1 RESULT err=0 tag=105 
> > nentries=0 etime=0
>  This is because ldif2db.pl just invokes an internal task using a task 
>  entry.  If you want to monitor the progress of the import, you'll have 
>  to look at the errors log, or use ldapsearch to query the entry that 
>  ldif2db.pl spits out (cn=import_2008_11_14_11_35_55, cn=import, 
>  cn=tasks, cn=config)
> >
> > If I repeat the operation I get 'operation error';  and if I try to 
> > access the directory, it appears to be completely empty.
>  Probably what happened is that you attempted to import from an LDIF 
>  file that did not contain the parent entries - the LDIF only contained 
>  your users and groups.  Import (ldif2db) is a _destructive_ operation 
>  - it will completely wipe out the contents of your database before 
>  adding the new entries.  In order to add an entry in LDAP, the parent 
>  entry must exist.   This means that if you want to import an LDIF 
>  file, and dc=lse,dc=ac,dc=uk is your base suffix, the LDIF file must 
>  contain the entry
>  dn: dc=lse,dc=ac,dc=uk
>  ...
>
>  and any other parent entries of the entries you want to import.
> >
> > So, two questions:
> >
> > - is this a reasonable way to go about this task, or are there other 
> > tools I should use?
> > - any suggestions for debugging?
> >
> > Thanks
> > Graham
> >
> >
> >
> >
> >
> > -- 
> > Fedora-directory-users mailing list
> > Fedora-directory-users(a)redhat.com
> > https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>  ------------------------------------------------------------------------
>
>  --
>  Fedora-directory-users mailing list
>  Fedora-directory-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>