How to restore replica admin in the master

[Mister Anonyme]

Hi,

I have two masters (in multi-master mode, they replicate each other) and 6 slaves.

I added a new schema file in /etc/dirsrv/slapd-XXX/schema and I restarted all dirsrv.  I
learned later that I had to stop the replication before adding a new schema file.  Because
of that, the netscaperoot seems to be corrupted because I wasn't able to do
replication between two masters.

So, I had to completely re-install two masters and re-import the database but is there a
way to re-configure the admin part of each replica (slave) servers ?  I could completely
re-install slaves too but if I can reconfigure the admin so I can see all replicas in the
Redhat Management Console, it would be nice.

Thank you!

_________________________________________________________________
New! Faster Messenger access on the new MSN homepage
http://go.microsoft.com/?linkid=9677406

[John A. Sullivan III]

On Thu, 2009-09-03 at 13:50 -0400, Mister Anonyme wrote:
>  Hi,
>  
>  I have two masters (in multi-master mode, they replicate each other)
>  and 6 slaves.
>  
>  I added a new schema file in /etc/dirsrv/slapd-XXX/schema and I
>  restarted all dirsrv.  I learned later that I had to stop the
>  replication before adding a new schema file.  Because of that, the
>  netscaperoot seems to be corrupted because I wasn't able to do
>  replication between two masters.
>  
>  So, I had to completely re-install two masters and re-import the
>  database but is there a way to re-configure the admin part of each
>  replica (slave) servers ?  I could completely re-install slaves too
>  but if I can reconfigure the admin so I can see all replicas in the
>  Redhat Management Console, it would be nice.
>   
<snip>
Ouch! I think I understand.  Unfortunately, I'm on the run and can't
explore it in detail but here is an excerpt from our internal
documentation on restoring the admin relationship between slave and
master and losing and then restoring the master from the slave database:

Once the data is restored, we need to tell LDAP1 that it is the
configuration master and that LDAP2 uses it.
On LDAP1 run "register-ds-admin.pl"
Then, on LDAP2 run "setup-ds-admin.pl -u" but, for some reason, it
insists on installing the CA cert and, since it already exists in the
database, it errors.  So we first remove the existing CA cert:
cd /etc/dirsrv/admin-serv
certutil -D -d . -n "CA certificate"
then run setup-ds-admin.pl -u and take defaults except we must enter the
path the to CA cert (/etc/dirsrv/admin-serv/MyCA.pem).

Hope this helps.  I think the original threads where Rich Megginson
helped us through this scenario are still in the archive.  Good luck -
John
-- 
John A. Sullivan III
Open Source Development Corporation
[phone]
jsullivan(a)opensourcedevel.com

http://www.spiritualoutreach.com
Making Christianity intelligible to secular society

[Mister Anonyme]

Hi,

I tried with setup-ds-admin.pl but the configuration files is already present so this
setup fails.  I forgot to add that I use the version 8.0.

Anyway, if I completely re-install two masters servers, configurations files for slaves
will be lost.  It seems that I don't have a choice to re-install slaves too.

As a final word, for those who use 8.0 and are using replication system, don't add a
new schema file in /etc/dirsrv/slapd-XXXX/schema.  I'll tell you why:

 I read the docs for DS 8.0 and anywhere it talks about add new schema file but I found it
myself by digging in /etc/dirsrv and I tested it in the lab.

Later, when I added a new bunch of users, I noticed that the replication was stopped
between two masters, but not between master and slaves.  I tried to understand why it
doesn't work anymore and I found out by reading in 8.1 (the next version that we
don't use it yet) documentation that it says that we need to stop all replication
before adding a new schema file.  

Heh, good to know, but it was already too late.

I tried everything like removing/creating replication agreement, removing local database,
recreate it, etc, the second master doesn't just want to start the replication. 
However, the replication between the first master and slaves is working well because I
first added a new schema file on the slave, the restarted the slapd.  After, I added it on
the first master, and then restarted it.  In fact, it worked very well until I added a new
bunch of users with the new attribute that's only present from the new schema file
that I added earlier.  Since then, the replication between two master just stopped, even
o=netscaperoot isn't replicated anymore.

The worst thing is, I first tried adding a new schema in the lab and it worked flawlessly,
even when I added some users.  I found out that the problem arise only when I restart
again one of two masters.  In other words, I stop the slapd, I add a new schema, I fire it
up.  I do the same thing on the second master.  It works.  I stop again the second, and
bam, you lost the replication and you just corrupted some database including the
o=netscaperoot.

So, be cautious when you add a new schema file ;-)


>  Subject: Re: [389-users] How to restore replica admin in the master
>  From: jsullivan(a)opensourcedevel.com
>  To: fedora-directory-users(a)redhat.com
>  Date: Thu, 3 Sep 2009 14:14:04 -0400
>  
>  On Thu, 2009-09-03 at 13:50 -0400, Mister Anonyme wrote:
>  > Hi,
>  > 
>  > I have two masters (in multi-master mode, they replicate each other)
>  > and 6 slaves.
>  > 
>  > I added a new schema file in /etc/dirsrv/slapd-XXX/schema and I
>  > restarted all dirsrv.  I learned later that I had to stop the
>  > replication before adding a new schema file.  Because of that, the
>  > netscaperoot seems to be corrupted because I wasn't able to do
>  > replication between two masters.
>  > 
>  > So, I had to completely re-install two masters and re-import the
>  > database but is there a way to re-configure the admin part of each
>  > replica (slave) servers ?  I could completely re-install slaves too
>  > but if I can reconfigure the admin so I can see all replicas in the
>  > Redhat Management Console, it would be nice.
>  > 
>  <snip>
>  Ouch! I think I understand.  Unfortunately, I'm on the run and can't
>  explore it in detail but here is an excerpt from our internal
>  documentation on restoring the admin relationship between slave and
>  master and losing and then restoring the master from the slave database:
>  
>  Once the data is restored, we need to tell LDAP1 that it is the
>  configuration master and that LDAP2 uses it.
>  On LDAP1 run "register-ds-admin.pl"
>  Then, on LDAP2 run "setup-ds-admin.pl -u" but, for some reason, it
>  insists on installing the CA cert and, since it already exists in the
>  database, it errors.  So we first remove the existing CA cert:
>  cd /etc/dirsrv/admin-serv
>  certutil -D -d . -n "CA certificate"
>  then run setup-ds-admin.pl -u and take defaults except we must enter the
>  path the to CA cert (/etc/dirsrv/admin-serv/MyCA.pem).
>  
>  Hope this helps.  I think the original threads where Rich Megginson
>  helped us through this scenario are still in the archive.  Good luck -
>  John
>  -- 
>  John A. Sullivan III
>  Open Source Development Corporation
>  [phone]
>  jsullivan(a)opensourcedevel.com
>  
>  http://www.spiritualoutreach.com
>  Making Christianity intelligible to secular society
>  
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users 
_________________________________________________________________
Click less, chat more: Messenger on MSN.ca
http://go.microsoft.com/?linkid=9677404

[Rich Megginson]

Mister Anonyme wrote:
>  Hi,
>
>  I tried with setup-ds-admin.pl but the configuration files is already 
>  present so this setup fails.  I forgot to add that I use the version 8.0. 
8.0 had a
problem in that it could not register a server with a remote 
configuration DS.  This problem has been fixed in 8.1.
>  Anyway, if I completely re-install two masters servers, configurations 
>  files for slaves will be lost.  It seems that I don't have a choice to 
>  re-install slaves too.
>
>  As a final word, for those who use 8.0 and are using replication 
>  system, don't add a new schema file in /etc/dirsrv/slapd-XXXX/schema.  
>  I'll tell you why:
>
>   I read the docs for DS 8.0 and anywhere it talks about add new schema 
>  file but I found it myself by digging in /etc/dirsrv and I tested it 
>  in the lab. 
If those docs need to be corrected, please send us the links.  Also
note 
that in 8.0:
If you want to add new schema to an existing instance, you must add the 
files to /etc/dirsrv/slapd-instancename/schema, then restart the server 
for the schema changes to take effect
/etc/dirsrv/schema is only for new instances only - existing servers 
don't use these files
schema files are not replicated - the only way to replicate schema is to 
add the new schema over LDAP

With 8.1 you have the ability to add schema files, then have the server 
reload them without having to restart the server, but the schema files 
added by copying them to the server instance schema directory will still 
not be replicated.
>  Later, when I added a new bunch of users, I noticed that the 
>  replication was stopped between two masters, but not between master 
>  and slaves.  I tried to understand why it doesn't work anymore 
Anything in the
errors or access logs?
>  and I found out by reading in 8.1 (the next version that we don't
> use 
>  it yet) documentation that it says that we need to stop all 
>  replication before adding a new schema file.  
Can you provide a link to the
documentation?
>  Heh, good to know, but it was already too late.
>
>  I tried everything like removing/creating replication agreement, 
>  removing local database, recreate it, etc, the second master doesn't 
>  just want to start the replication.  However, the replication between 
>  the first master and slaves is working well because I first added a 
>  new schema file on the slave, the restarted the slapd.  After, I added 
>  it on the first master, and then restarted it.  In fact, it worked 
>  very well until I added a new bunch of users with the new attribute 
>  that's only present from the new schema file that I added earlier.  
>  Since then, the replication between two master just stopped, even 
>  o=netscaperoot isn't replicated anymore.
>
>  The worst thing is, I first tried adding a new schema in the lab and 
>  it worked flawlessly, even when I added some users.  I found out that 
>  the problem arise only when I restart again one of two masters.  In 
>  other words, I stop the slapd, I add a new schema, I fire it up.  I do 
>  the same thing on the second master.  It works.  I stop again the 
>  second, and bam, you lost the replication and you just corrupted some 
>  database including the o=netscaperoot. 
I'm not really sure what's going on
here.  I seriously doubt there is 
any data corruption happening (unless there is some disk/hardware 
failure).  I would first suggest you check your errors log in 
/var/log/dirsrv/slapd-instancename/errors
>  So, be cautious when you add a new schema file ;-)
>
>
>  > Subject: Re: [389-users] How to restore replica admin in the master
>  > From: jsullivan(a)opensourcedevel.com
>  > To: fedora-directory-users(a)redhat.com
>  > Date: Thu, 3 Sep 2009 14:14:04 -0400
>  >
>  > On Thu, 2009-09-03 at 13:50 -0400, Mister Anonyme wrote:
>  > > Hi,
>  > >
>  > > I have two masters (in multi-master mode, they replicate each other)
>  > > and 6 slaves.
>  > >
>  > > I added a new schema file in /etc/dirsrv/slapd-XXX/schema and I
>  > > restarted all dirsrv. I learned later that I had to stop the
>  > > replication before adding a new schema file. Because of that, the
>  > > netscaperoot seems to be corrupted because I wasn't able to do
>  > > replication between two masters.
>  > >
>  > > So, I had to completely re-install two masters and re-import the
>  > > database but is there a way to re-configure the admin part of each
>  > > replica (slave) servers ? I could completely re-install slaves too
>  > > but if I can reconfigure the admin so I can see all replicas in the
>  > > Redhat Management Console, it would be nice.
>  > >
>  > <snip>
>  > Ouch! I think I understand. Unfortunately, I'm on the run and can't
>  > explore it in detail but here is an excerpt from our internal
>  > documentation on restoring the admin relationship between slave and
>  > master and losing and then restoring the master from the slave database:
>  >
>  > Once the data is restored, we need to tell LDAP1 that it is the
>  > configuration master and that LDAP2 uses it.
>  > On LDAP1 run "register-ds-admin.pl"
>  > Then, on LDAP2 run "setup-ds-admin.pl -u" but, for some reason, it
>  > insists on installing the CA cert and, since it already exists in the
>  > database, it errors. So we first remove the existing CA cert:
>  > cd /etc/dirsrv/admin-serv
>  > certutil -D -d . -n "CA certificate"
>  > then run setup-ds-admin.pl -u and take defaults except we must enter the
>  > path the to CA cert (/etc/dirsrv/admin-serv/MyCA.pem).
>  >
>  > Hope this helps. I think the original threads where Rich Megginson
>  > helped us through this scenario are still in the archive. Good luck -
>  > John
>  > --
>  > John A. Sullivan III
>  > Open Source Development Corporation
>  > [phone]
>  > jsullivan(a)opensourcedevel.com
>  >
>  > http://www.spiritualoutreach.com
>  > Making Christianity intelligible to secular society
>  >
>  > --
>  > 389 users mailing list
>  > 389-users(a)redhat.com
>  > https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>  ------------------------------------------------------------------------
>  Faster Hotmail access now on the new MSN homepage. 
>  <http://go.microsoft.com/?linkid=9677399>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

[Mister Anonyme]

>  Date: Thu, 3 Sep 2009 13:30:30 -0600
>  From: rmeggins(a)redhat.com
>  To: fedora-directory-users(a)redhat.com
>  Subject: Re: [389-users] How to restore replica admin in the master 

>  If those docs need to be corrected, please send us the links.  Also
> note 
>  that in 8.0:
>  If you want to add new schema to an existing instance, you must add the 
>  files to /etc/dirsrv/slapd-instancename/schema, then restart the server 
>  for the schema changes to take effect
>  /etc/dirsrv/schema is only for new instances only - existing servers 
>  don't use these files
>  schema files are not replicated - the only way to replicate schema is to 
>  add the new schema over LDAP 
I read docs from here:

http://www.redhat.com/docs/manuals/dir-server/

About schemas, I read here:

http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Comma...

And here:
http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Extending_the_Direct...



I just can't find the description you just put here ?  It must be hidden in some
obscure area... or I need eyeglasses.



>  With 8.1 you have the ability to add schema files, then have the
> server 
>  reload them without having to restart the server, but the schema files 
>  added by copying them to the server instance schema directory will still 
>  not be replicated. 

Yep exactly.


>  >
>  > Later, when I added a new bunch of users, I noticed that the 
>  > replication was stopped between two masters, but not between master 
>  > and slaves.  I tried to understand why it doesn't work anymore
>  Anything in the errors or access logs? 
Yep, it happens each time I add a new schema on a replicated system.  Here are the logs:


Master A:

[02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin -
agmt="cn=INSTANCE_prod" (SERVER:389): Unable to acquire replica:
there is no replicated area "dc=name,dc=domain,dc=net" on the consumer
server. Replication is aborting. 
[02/Sep/2009:10:15:17 -0400]
NSMMReplicationPlugin - agmt="cn=INSTANCE_prod" (SERVER:389):
Incremental update failed and requires administrator action
[02/Sep/2009:11:44:09 -0400] NSMMReplicationPlugin -
agmt="cn=INSTANCE_netscaperoot" (SERVER:389): Unable to acquire
replica: there is no replicated area "o=netscaperoot" on the consumer
server. Replication is aborting. 
[02/Sep/2009:11:44:09 -0400]
NSMMReplicationPlugin - agmt="cn=INSTANCE_netscaperoot" (SERVER:389):
Incremental update failed and requires administrator action


Master B:

[02/Sep/2009:11:15:18 -0400] NSMMReplicationPlugin - conn=73 op=3
replica="unknown": Unable to acquire replica: error: no such replica
[02/Sep/2009:11:44:10 -0400] NSMMReplicationPlugin - conn=3572 op=3
replica="unknown": Unable to acquire replica: error: no such replica

Take note that it happens only when I add a new schema and I restart the server.  When I
restart without adding a new schema, I don't have that kind of error, it just works. 


What I did is I copy the schema in /etc/dirsrv/slapd-XXXX/schema and then I restart the
server.  

However, in the lab, at the installation, I initially copied the schema (before the the
start of the replication) and started both servers and it works flawlessly.


>  > and I found out by reading in 8.1 (the next version that we
> don't use 
>  > it yet) documentation that it says that we need to stop all 
>  > replication before adding a new schema file. 
>  Can you provide a link to the documentation? 
There you go:

http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloa...


>  I'm not really sure what's going on here.  I seriously doubt
> there is 
>  any data corruption happening (unless there is some disk/hardware 
>  failure).  I would first suggest you check your errors log in 
>  /var/log/dirsrv/slapd-instancename/errors 

Maybe ? I find it very weird too but the fact is: I'm able to reproduce the issue in
the lab.  More than one.  I already verified the logs and I also enabled the verbose mode
by doing this:

dn: cn=config
changetype: modify
replace: nsslapd-errorlog-level
nsslapd-errorlog-level: 8192

Thanks!



_________________________________________________________________
New! Open Messenger faster on the MSN homepage
http://go.microsoft.com/?linkid=9677405

[Rich Megginson]

Mister Anonyme wrote:
>  > Date: Thu, 3 Sep 2009 13:30:30 -0600
>  > From: rmeggins(a)redhat.com
>  > To: fedora-directory-users(a)redhat.com
>  > Subject: Re: [389-users] How to restore replica admin in the master
>
>
>  > If those docs need to be corrected, please send us the links.  Also note 
>  > that in 8.0:
>  > If you want to add new schema to an existing instance, you must add the 
>  > files to /etc/dirsrv/slapd-instancename/schema, then restart the server 
>  > for the schema changes to take effect
>  > /etc/dirsrv/schema is only for new instances only - existing servers 
>  > don't use these files
>  > schema files are not replicated - the only way to replicate schema is to 
>  > add the new schema over LDAP
>
>  I read docs from here:
>
>  http://www.redhat.com/docs/manuals/dir-server/
> <%20%20http://www.redhat.com/docs/manuals/dir-server/>
>    
>
>  About schemas, I read here:
>
> http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Comma...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Command_File_Reference-Core_Server_Configuration_Reference.html#Configuration_Command_File_Reference-Server_Configuration___Overview-LDIF_Configuration_Files___Location>
2.1.1 is incorrect

https://bugzilla.redhat.com/show_bug.cgi?id=521139  Bug 521139 -  
incorrect config and schema file location

This is also incorrect in the 8.1 Schema Reference (although it is a 
different incorrect location);
https://bugzilla.redhat.com/show_bug.cgi?id=521140  Bug 521140 -  
incorrect schema file location

Thank your for reporting this bug - it should be fixed shortly.

Other than that, it doesn't really talk about how to extend the schema 
with schema files.  I'm not really sure where it talks about that.

>  And here:
>  http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Extending_the_Direct...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Extending_the_Directory_Schema.html>
This section describes how to manage the schema using the console - it 
says nothing about using schema files.
>  I just can't find the description you just put here ?  It must be 
>  hidden in some obscure area... or I need eyeglasses. 
I don't know if it is
documented.
>  > With 8.1 you have the ability to add schema files, then have the server
>  > reload them without having to restart the server, but the schema files
>  > added by copying them to the server instance schema directory will 
>  still
>  > not be replicated.
>
>
>  Yep exactly.
>
>
>  > >
>  > > Later, when I added a new bunch of users, I noticed that the
>  > > replication was stopped between two masters, but not between master
>  > > and slaves. I tried to understand why it doesn't work anymore
>  > Anything in the errors or access logs?
>
>  Yep, it happens each time I add a new schema on a replicated system.  
>  Here are the logs:
>
>
>  Master A:
>
>  [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_prod" (SERVER:389): Unable to acquire replica: there 
>  is no replicated area "dc=name,dc=domain,dc=net" on the consumer 
>  server. Replication is aborting. 
no replicated area
"dc=name,dc=domain,dc=net" on the consumer server
This means something has broken or removed the replication 
configuration.  A schema file should not be able to do that.
>  [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_prod" (SERVER:389): Incremental update failed and 
>  requires administrator action
>  [02/Sep/2009:11:44:09 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_netscaperoot" (SERVER:389): Unable to acquire 
>  replica: there is no replicated area "o=netscaperoot" on the consumer 
>  server. Replication is aborting.
>  [02/Sep/2009:11:44:09 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_netscaperoot" (SERVER:389): Incremental update 
>  failed and requires administrator action
>
>
>  Master B:
>
>  [02/Sep/2009:11:15:18 -0400] NSMMReplicationPlugin - conn=73 op=3 
>  replica="unknown": Unable to acquire replica: error: no such replica
>  [02/Sep/2009:11:44:10 -0400] NSMMReplicationPlugin - conn=3572 op=3 
>  replica="unknown": Unable to acquire replica: error: no such replica
>
>  Take note that it happens only when I add a new schema and I restart 
>  the server.  When I restart without adding a new schema, I don't have 
>  that kind of error, it just works. 
>
>  What I did is I copy the schema in /etc/dirsrv/slapd-XXXX/schema and 
>  then I restart the server.  
Can you post your schema file?
>  However, in the lab, at the installation, I initially copied the 
>  schema (before the the start of the replication) and started both 
>  servers and it works flawlessly.
>
>
>  > > and I found out by reading in 8.1 (the next version that we don't use
>  > > it yet) documentation that it says that we need to stop all
>  > > replication before adding a new schema file.
>  > Can you provide a link to the documentation?
>
>  There you go:
>
> http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloa...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloading-schema.html#reloading-schema-with-replication>
>
>
>  > I'm not really sure what's going on here. I seriously doubt there is
>  > any data corruption happening (unless there is some disk/hardware
>  > failure). I would first suggest you check your errors log in
>  > /var/log/dirsrv/slapd-instancename/errors
>
>
>  Maybe ? I find it very weird too but the fact is: I'm able to 
>  reproduce the issue in the lab.  More than one.  I already verified 
>  the logs and I also enabled the verbose mode by doing this:
>
>  dn: cn=config
>  changetype: modify
>  replace: nsslapd-errorlog-level
>  nsslapd-errorlog-level: 8192
>
>    
>  Thanks!
>
>
>
>  ------------------------------------------------------------------------
>  New! Open Hotmail faster on the new MSN homepage! 
>  <http://go.microsoft.com/?linkid=9677400>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

[Mister Anonyme]

>  Date: Thu, 3 Sep 2009 14:29:33 -0600
>  From: rmeggins(a)redhat.com
>  To: fedora-directory-users(a)redhat.com
>  Subject: Re: [389-users] How to restore replica admin in the master
>  
>  >
>  > [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin - 
>  > agmt="cn=INSTANCE_prod" (SERVER:389): Unable to acquire replica: there 
>  > is no replicated area "dc=name,dc=domain,dc=net" on the consumer 
>  > server. Replication is aborting. 
>  no replicated area "dc=name,dc=domain,dc=net" on the
> consumer server
>  This means something has broken or removed the replication 
>  configuration.  A schema file should not be able to do that. 
I know, myself, I don't understand how a schema file could break it.  I removed all
replication agreements and re-created and those errors are still present until a do a
complete reinstallation of both DS master servers.


>  > What I did is I copy the schema in /etc/dirsrv/slapd-XXXX/schema
> and 
>  > then I restart the server.  
>  Can you post your schema file?
>  
See attached file.



_________________________________________________________________
New: Messenger sign-in on the MSN homepage
http://go.microsoft.com/?linkid=9677403

[Rich Megginson]

Mister Anonyme wrote:
>  > Date: Thu, 3 Sep 2009 14:29:33 -0600
>  > From: rmeggins(a)redhat.com
>  > To: fedora-directory-users(a)redhat.com
>  > Subject: Re: [389-users] How to restore replica admin in the master
>  >
>  > >
>  > > [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin -
>  > > agmt="cn=INSTANCE_prod" (SERVER:389): Unable to acquire replica: 
>  there
>  > > is no replicated area "dc=name,dc=domain,dc=net" on the consumer
>  > > server. Replication is aborting.
>
>  > no replicated area "dc=name,dc=domain,dc=net" on the consumer server
>  > This means something has broken or removed the replication
>  > configuration. A schema file should not be able to do that.
>
>  I know, myself, I don't understand how a schema file could break it.  
>  I removed all replication agreements and re-created and those errors 
>  are still present until a do a complete reinstallation of both DS 
>  master servers. 
Let me see if I understand what's going on.  You copy a schema
file to 
the /etc/dirsrv/slapd-instance/schema directory on the supplier, restart 
the supplier, and you get that error from the consumer?  What's in the 
consumer error log?  Do you have a cn=replica, 
cn="dc=name,dc=domain,dc=net", cn=mapping tree, cn-config entry in the 
consumer dse.ldif?
>  > > What I did is I copy the schema in /etc/dirsrv/slapd-XXXX/schema and
>  > > then I restart the server.
>
>  > Can you post your schema file?
>  >
>
>  See attached file. 
Seems ok - unlikely to be the culprit . . .
>  ------------------------------------------------------------------------
>  Less clicking: Hotmail access on the new MSN homepage. 
>  <http://go.microsoft.com/?linkid=9677398>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

[Mister Anonyme]

>  Date: Thu, 3 Sep 2009 15:30:19 -0600
>  From: rmeggins(a)redhat.com
>  To: fedora-directory-users(a)redhat.com
>  Subject: Re: [389-users] How to restore replica admin in the master 
>  > See attached file.
>  Seems ok - unlikely to be the culprit . . .
>  > 
You're right.  I found out that the schema was just a coincidence.

I found out that it was the replication was working will until I restart one of two
masters.  In fact, I never restarted it since it was up.

Here's the file.inf on the second master, at the installation:

[General]
AdminDomain = domain.net
SuiteSpotGroup = nobody
ConfigDirectoryLdapURL = ldap://MASTERA:389/o=NetscapeRoot
ConfigDirectoryAdminID = admin
FullMachineName = SERVER
SuiteSpotUserID = nobody
ConfigDirectoryAdminPwd = PASS

[admin]
ServerAdminID = admin
ServerAdminPwd = pass
SysUser = nobody
Port = 9830

[slapd]
InstallLdifFile = suggest
ServerIdentifier = LOCALSERVER
ServerPort = 389
AddOrgEntries = Yes
RootDN = cn=Directory Manager
RootDNPwd = PASS
Suffix = dc=bd,dc=domain,dc=net
UseExistingMC = Yes
AddSampleEntries = No
ConfigFile = netscaperootdb.ldif
ConfigFile = devdb.ldif
ConfigFile = proddb.ldif
ConfigFile = repluser.ldif
ConfigFile = changelog.ldif
ConfigFile = replica.ldif
ConfigFile = replagreement.ldif

What I did is I removed all ConfigFile lines and I simply executed ldapmodify on each file
after the /usr/sbin/setup-ds.pl (I had to modify each file by adding "changetype:
add").  It seems to fix the issue.  I added a new schema and replication is still
working even if the other master doesn't have it.  I restarted both masters, and it
still works.

It seems it break the "implementation" if I use ConfigFile directly in the
file.inf at the installation.  That's why I was able to "reproduce" the
issue because I was using the same installation method.



_________________________________________________________________
New: Messenger sign-in on the MSN homepage
http://go.microsoft.com/?linkid=9677403

[Mister Anonyme]

I forgot to add errors files.  As you can see, it works until I restart the server.  I
took a look in dse.ldif and they are present.


From: benetage(a)hotmail.com
To: fedora-directory-users(a)redhat.com
Subject: RE: [389-users] How to restore replica admin in the master
Date: Fri, 4 Sep 2009 08:21:29 -0400









>  Date: Thu, 3 Sep 2009 15:30:19 -0600
>  From: rmeggins(a)redhat.com
>  To: fedora-directory-users(a)redhat.com
>  Subject: Re: [389-users] How to restore replica admin in the master 
>  > See attached file.
>  Seems ok - unlikely to be the culprit . . .
>  > 
You're right.  I found out that the schema was just a coincidence.

I found out that it was the replication was working will until I restart one of two
masters.  In fact, I never restarted it since it was up.

Here's the file.inf on the second master, at the installation:

[General]
AdminDomain = domain.net
SuiteSpotGroup = nobody
ConfigDirectoryLdapURL = ldap://MASTERA:389/o=NetscapeRoot
ConfigDirectoryAdminID = admin
FullMachineName = SERVER
SuiteSpotUserID = nobody
ConfigDirectoryAdminPwd = PASS

[admin]
ServerAdminID = admin
ServerAdminPwd = pass
SysUser = nobody
Port = 9830

[slapd]
InstallLdifFile = suggest
ServerIdentifier = LOCALSERVER
ServerPort = 389
AddOrgEntries = Yes
RootDN = cn=Directory Manager
RootDNPwd = PASS
Suffix = dc=bd,dc=domain,dc=net
UseExistingMC = Yes
AddSampleEntries = No
ConfigFile = netscaperootdb.ldif
ConfigFile = devdb.ldif
ConfigFile = proddb.ldif
ConfigFile = repluser.ldif
ConfigFile = changelog.ldif
ConfigFile = replica.ldif
ConfigFile = replagreement.ldif

What I did is I removed all ConfigFile lines and I simply executed ldapmodify on each file
after the /usr/sbin/setup-ds.pl (I had to modify each file by adding "changetype:
add").  It seems to fix the issue.  I added a new schema and replication is still
working even if the other master doesn't have it.  I restarted both masters, and it
still works.

It seems it break the "implementation" if I use ConfigFile directly in the
file.inf at the installation.  That's why I was able to "reproduce" the
issue because I was using the same installation method.



Less clicking: Hotmail access on the new MSN homepage.
_________________________________________________________________
New! Get to Messenger faster: Sign-in here now!
http://go.microsoft.com/?linkid=9677407

[Rich Megginson]

Mister Anonyme wrote:
>  > Date: Thu, 3 Sep 2009 15:30:19 -0600
>  > From: rmeggins(a)redhat.com
>  > To: fedora-directory-users(a)redhat.com
>  > Subject: Re: [389-users] How to restore replica admin in the master
>
>  > > See attached file.
>  > Seems ok - unlikely to be the culprit . . .
>  > >
>
>  You're right.  I found out that the schema was just a coincidence.
>
>  I found out that it was the replication was working will until I 
>  restart one of two masters.  In fact, I never restarted it since it 
>  was up.
>
>  Here's the file.inf on the second master, at the installation:
>
>  [General]
>  AdminDomain = domain.net
>  SuiteSpotGroup = nobody
>  ConfigDirectoryLdapURL = ldap://MASTERA:389/o=NetscapeRoot
>  ConfigDirectoryAdminID = admin
>  FullMachineName = SERVER
>  SuiteSpotUserID = nobody
>  ConfigDirectoryAdminPwd = PASS
>
>  [admin]
>  ServerAdminID = admin
>  ServerAdminPwd = pass
>  SysUser = nobody
>  Port = 9830
>
>  [slapd]
>  InstallLdifFile = suggest
>  ServerIdentifier = LOCALSERVER
>  ServerPort = 389
>  AddOrgEntries = Yes
>  RootDN = cn=Directory Manager
>  RootDNPwd = PASS
>  Suffix = dc=bd,dc=domain,dc=net
>  UseExistingMC = Yes 
You also need
SlapdConfigForMC = No
not sure if that's causing the problems
>  AddSampleEntries = No
>  ConfigFile = netscaperootdb.ldif
>  ConfigFile = devdb.ldif
>  ConfigFile = proddb.ldif
>  ConfigFile = repluser.ldif
>  ConfigFile = changelog.ldif
>  ConfigFile = replica.ldif
>  ConfigFile = replagreement.ldif
>
>  What I did is I removed all ConfigFile lines and I simply executed 
>  ldapmodify on each file after the /usr/sbin/setup-ds.pl (I had to 
>  modify each file by adding "changetype: add").  It seems to fix the 
>  issue.  I added a new schema and replication is still working even if 
>  the other master doesn't have it.  I restarted both masters, and it 
>  still works. 
I would have to review all of these ldif files - one or more of them
are 
causing problems.
>  It seems it break the "implementation" if I use ConfigFile directly in 
>  the file.inf at the installation.  That's why I was able to 
>  "reproduce" the issue because I was using the same installation method.
>
>
>
>  ------------------------------------------------------------------------
>  Less clicking: Hotmail access on the new MSN homepage. 
>  <http://go.microsoft.com/?linkid=9677398>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

[Rich Megginson]

Mister Anonyme wrote:
>  > Date: Thu, 3 Sep 2009 13:30:30 -0600
>  > From: rmeggins(a)redhat.com
>  > To: fedora-directory-users(a)redhat.com
>  > Subject: Re: [389-users] How to restore replica admin in the master
>
>
>  > If those docs need to be corrected, please send us the links.  Also note 
>  > that in 8.0:
>  > If you want to add new schema to an existing instance, you must add the 
>  > files to /etc/dirsrv/slapd-instancename/schema, then restart the server 
>  > for the schema changes to take effect
>  > /etc/dirsrv/schema is only for new instances only - existing servers 
>  > don't use these files
>  > schema files are not replicated - the only way to replicate schema is to 
>  > add the new schema over LDAP
>
>  I read docs from here:
>
>  http://www.redhat.com/docs/manuals/dir-server/
> <%20%20http://www.redhat.com/docs/manuals/dir-server/>
>    
>
>  About schemas, I read here:
>
> http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Comma...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Command_File_Reference-Core_Server_Configuration_Reference.html#Configuration_Command_File_Reference-Server_Configuration___Overview-LDIF_Configuration_Files___Location>
>
>  And here:
>  http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Extending_the_Direct...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/ag/8.0/Extending_the_Directory_Schema.html>
>
>
>
>  I just can't find the description you just put here ?  It must be 
>  hidden in some obscure area... or I need eyeglasses.
>
>
>
>  > With 8.1 you have the ability to add schema files, then have the server
>  > reload them without having to restart the server, but the schema files
>  > added by copying them to the server instance schema directory will 
>  still
>  > not be replicated.
>
>
>  Yep exactly.
>
>
>  > >
>  > > Later, when I added a new bunch of users, I noticed that the
>  > > replication was stopped between two masters, but not between master
>  > > and slaves. I tried to understand why it doesn't work anymore
>  > Anything in the errors or access logs?
>
>  Yep, it happens each time I add a new schema on a replicated system.  
>  Here are the logs:
>
>
>  Master A:
>
>  [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_prod" (SERVER:389): Unable to acquire replica: there 
>  is no replicated area "dc=name,dc=domain,dc=net" on the consumer 
>  server. Replication is aborting.
>  [02/Sep/2009:10:15:17 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_prod" (SERVER:389): Incremental update failed and 
>  requires administrator action
>  [02/Sep/2009:11:44:09 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_netscaperoot" (SERVER:389): Unable to acquire 
>  replica: there is no replicated area "o=netscaperoot" on the consumer 
>  server. Replication is aborting.
>  [02/Sep/2009:11:44:09 -0400] NSMMReplicationPlugin - 
>  agmt="cn=INSTANCE_netscaperoot" (SERVER:389): Incremental update 
>  failed and requires administrator action
>
>
>  Master B:
>
>  [02/Sep/2009:11:15:18 -0400] NSMMReplicationPlugin - conn=73 op=3 
>  replica="unknown": Unable to acquire replica: error: no such replica
>  [02/Sep/2009:11:44:10 -0400] NSMMReplicationPlugin - conn=3572 op=3 
>  replica="unknown": Unable to acquire replica: error: no such replica
>
>  Take note that it happens only when I add a new schema and I restart 
>  the server.  When I restart without adding a new schema, I don't have 
>  that kind of error, it just works. 
>
>  What I did is I copy the schema in /etc/dirsrv/slapd-XXXX/schema and 
>  then I restart the server. 
>
>  However, in the lab, at the installation, I initially copied the 
>  schema (before the the start of the replication) and started both 
>  servers and it works flawlessly.
>
>
>  > > and I found out by reading in 8.1 (the next version that we don't use
>  > > it yet) documentation that it says that we need to stop all
>  > > replication before adding a new schema file.
>  > Can you provide a link to the documentation?
>
>  There you go:
>
> http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloa...
>
>
> <%20http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloading-schema.html#reloading-schema-with-replication>
How did you stop and restart replication?
>  > I'm not really sure what's going on here. I seriously doubt there is
>  > any data corruption happening (unless there is some disk/hardware
>  > failure). I would first suggest you check your errors log in
>  > /var/log/dirsrv/slapd-instancename/errors
>
>
>  Maybe ? I find it very weird too but the fact is: I'm able to 
>  reproduce the issue in the lab.  More than one.  I already verified 
>  the logs and I also enabled the verbose mode by doing this:
>
>  dn: cn=config
>  changetype: modify
>  replace: nsslapd-errorlog-level
>  nsslapd-errorlog-level: 8192
>
>    
>  Thanks!
>
>
>
>  ------------------------------------------------------------------------
>  New! Open Hotmail faster on the new MSN homepage! 
>  <http://go.microsoft.com/?linkid=9677400>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

[Mister Anonyme]

>  Date: Thu, 3 Sep 2009 14:31:20 -0600
>  From: rmeggins(a)redhat.com
>  To: fedora-directory-users(a)redhat.com
>  Subject: Re: [389-users] How to restore replica admin in the master 
>  > There you go:
>  >
> http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloa...
>
>  >
> <%20http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloading-schema.html#reloading-schema-with-replication>
>  How did you stop and restart replication? 

I didn't try it.  Actually, I copied the schema file and then finished the
installation of the Multi-Master Replication.

I don't think we can simply stop the replication on a running MMR on DS 8.0, I think
we need to remove all replication agreements before adding a new schema file.  Maybe with
DS 8.1 we can stop it ? 

_________________________________________________________________
Click less, chat more: Messenger on MSN.ca
http://go.microsoft.com/?linkid=9677404

[Rich Megginson]

Mister Anonyme wrote:
>  > Date: Thu, 3 Sep 2009 14:31:20 -0600
>  > From: rmeggins(a)redhat.com
>  > To: fedora-directory-users(a)redhat.com
>  > Subject: Re: [389-users] How to restore replica admin in the master
>
>  > > There you go:
>  > > 
>
> http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloa...
>
>
>  > > 
>
> <%20http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dynamically-reloading-schema.html#reloading-schema-with-replication>
>
>  > How did you stop and restart replication?
>
>
>  I didn't try it.  Actually, I copied the schema file and then finished 
>  the installation of the Multi-Master Replication.
>
>  I don't think we can simply stop the replication on a running MMR on 
>  DS 8.0, I think we need to remove all replication agreements before 
>  adding a new schema file. 
No, you should not need to remove the replication
agreements.  With 8.0, 
you should just be able to add the schema file to 
/etc/dirsrv/slapd-instance/schema and restart the server.  With 8.1, you 
add the schema file to /etc/dirsrv/slapd-instance/schema and run the 
schema reload task.
>  Maybe with DS 8.1 we can stop it ?
>
>  ------------------------------------------------------------------------
>  Faster Hotmail access now on the new MSN homepage. 
>  <http://go.microsoft.com/?linkid=9677399>
>  ------------------------------------------------------------------------
>
>  --
>  389 users mailing list
>  389-users(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-directory-users
>