Replication conflict woes

[John Bowman]

After a lot of patching in order to get the environment up to date in order
to add a new CA replica and remove our IPA 3.0 servers we ended up with a
bunch of conflicts and other inconsistencies:

$ ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa0.domain.tld" -D
"cn=directory manager" -w secret -b "dc=domain,dc=tld"
"nsds5ReplConflict=*" \ nsds5ReplConflict
dn:
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
dn:
cn=ipaservers+nsuniqueid=e8d2f707-512111e7-9205b5bf-43202000,cn=ng,cn=alt,dc=domain,dc=tld
dn:
cn=domain+nsuniqueid=e8d2f70e-512111e7-9205b5bf-43202000,cn=topology,cn=ipa,cn=etc,dc=domain,dc=tld
dn:
cn=locations+nsuniqueid=e8d2f712-512111e7-9205b5bf-43202000,cn=etc,dc=domain,dc=tld
dn: cn=DNS
Administrators+nsuniqueid=e8d2f718-512111e7-9205b5bf-43202000,cn=privileges,cn=pbac,dc=domain,dc=tld
dn: cn=DNS
Servers+nsuniqueid=e8d2f71a-512111e7-9205b5bf-43202000,cn=privileges,cn=pbac,dc=domain,dc=tld
dn:
cn=cas+nsuniqueid=e8d2f71c-512111e7-9205b5bf-43202000,cn=ca,dc=domain,dc=tld
dn:
cn=dogtag+nsuniqueid=e8d2f74d-512111e7-9205b5bf-43202000,cn=custodia,cn=ipa,cn=etc,dc=domain,dc=tld
dn:
cn=ca+nsuniqueid=e8d2f750-512111e7-9205b5bf-43202000,cn=topology,cn=ipa,cn=etc,dc=domain,dc=tld
dn: cn=System: Add
CA+nsuniqueid=e8d2f75d-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Delete
CA+nsuniqueid=e8d2f761-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Modify
CA+nsuniqueid=e8d2f765-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Read
CAs+nsuniqueid=e8d2f769-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Modify DNS Servers
Configuration+nsuniqueid=e8d2f77a-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Read DNS Servers
Configuration+nsuniqueid=e8d2f77e-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Add IPA
Locations+nsuniqueid=e8d2f807-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Modify IPA
Locations+nsuniqueid=e8d2f80b-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Read IPA
Locations+nsuniqueid=e8d2f80f-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Remove IPA
Locations+nsuniqueid=e8d2f813-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Read Locations of IPA
Servers+nsuniqueid=e8d2f82c-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Read Status of Services on IPA
Servers+nsuniqueid=e8d2f830-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Manage Service
Principals+nsuniqueid=e8d2f834-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn: cn=System: Manage User
Principals+nsuniqueid=e8d2f866-512111e7-9205b5bf-43202000,cn=permissions,cn=pbac,dc=domain,dc=tld
dn:
dnaHostname=ipa1.domain.tld+dnaPortNum=0+nsuniqueid=c90407a3-51e311e7-9205b5bf-43202000,cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=domain,dc=tld


Looking only at the first one I see two entries for it:
$ ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa0.domain.tld" -D
"cn=directory manager" -w secret -b
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
-s base
dn:
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
ipaUniqueID: fe7226e4-5121-11e7-82f1-005056972fd9
cn: ipaservers
description: IPA server hosts
objectClass: top
objectClass: ipahostgroup
objectClass: ipaobject
objectClass: groupOfNames
objectClass: nestedGroup
objectClass: mepOriginEntry
mepManagedEntry: cn=ipaservers,cn=ng,cn=alt,dc=domain,dc=tld
member: fqdn=ipa1.domain.tld,cn=computers,cn=accounts,dc=domain,dc=tld

[jbowman@idm ipa_check_consistency]$ ldapsearch -o ldif-wrap=no -ZZ -LLLx
-h "ipa0.domain.tld" -D "cn=directory manager" -w secret -b
cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld -s base
dn: cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld
ipaUniqueID: 319cb1ce-c21b-11e6-bab9-005056977521
cn: ipaservers
description: IPA server hosts
objectClass: top
objectClass: ipahostgroup
objectClass: ipaobject
objectClass: groupOfNames
objectClass: nestedGroup
objectClass: mepOriginEntry
mepManagedEntry: cn=ipaservers,cn=ng,cn=alt,dc=domain,dc=tld
member: fqdn=ipa1.domain.tld,cn=computers,cn=accounts,dc=domain,dc=tld
member: fqdn=ipa4.domain.tld,cn=computers,cn=accounts,dc=domain,dc=tld
member: fqdn=ipa5.domain.tld,cn=computers,cn=accounts,dc=domain,dc=tld
memberOf: cn=replication
administrators,cn=privileges,cn=pbac,dc=domain,dc=tld
memberOf: cn=add replication
agreements,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=modify replication
agreements,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=remove replication
agreements,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=read passsync managers
configuration,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=modify passsync managers
configuration,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=read ldbm database
configuration,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=add configuration
sub-entries,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=modify dna range,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=read dna range,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf: cn=read replication
agreements,cn=permissions,cn=pbac,dc=domain,dc=tld
memberOf:
ipauniqueid=87c611a4-3753-11e3-a382-0050568e07ed,cn=sudorules,cn=sudo,dc=domain,dc=tld
memberOf: cn=ipaservers,cn=ng,cn=alt,dc=domain,dc=tld
memberOf:
cn=ipaservers+nsuniqueid=e8d2f707-512111e7-9205b5bf-43202000,cn=ng,cn=alt,dc=domain,dc=tld

I made the mistake of trying to delete:
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld

After a successful deletion with ldapmodify, the entry is removed on 5 of
the 6 servers but 1 server (in this case ipa1.domain.tld) it deletes the
valid entry on that server.   I'm concerned these errors could cause other
issues further down the road and would like to get them cleared up but not
having much success which doesn't build confidence unfortunately.  Any tips
would be appreciated.

If it helps ipa0 = RHEL 6 with IPA 3.0
                ipa1 = RHEL 7 with IPA 4.4 (recently updated from 4.2)
                ipa2 = RHEL 6 with IPA 3.0
                ipa3 = RHEL 6 with IPA 3.0
                ipa4 = RHEL 7 with IPA 4.4
                ipa5 = RHEL 7 with IPA 4.4


Thanks!

Here is a specific example:

conflict entry:  dn:
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld

Step 1:
$ ldapmodify -D "cn=directory manager" -w secret -p 389 -h ipa0.domain.tld
dn:
cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
changetype: modrdn
newrdn: cn=ipaservtemp
deleteoldrdn: 0
modifying rdn of entry
"cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld"

Step 2:

$ ldapmodify -x -D "cn=directory manager" -w secret -p 389 -h ipa1.domain.tld
dn: cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld
changetype: modify
delete: cn
cn: ipaservers

delete: nsds5ReplConflict

Step 3:
$ ldapmodify -x -D "cn=directory manager" -w secret -p 389 -h ipa1.domain.tld
dn: cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld
changetype: modrdn
newrdn: cn=ipaservers
deleteoldrdn: 1
modifying rdn of entry
"cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld"

This produces the following error:
ldap_rename: Operations error (1)

When I check for the conflict its gone but on ipa1.domain.tld it gives me an error saying
it can't find ipaservers:
$ ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa1.domain.tld" -D
"cn=directory manager" -w secret -b
cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld -s base
No such object (32)
Matched DN: cn=hostgroups,cn=accounts,dc=domain,dc=tld

But when I list all the hostgroups on that same server it does show up:
ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa1.domain.tld" -D "cn=directory
manager" -w secret -b cn=hostgroups,cn=accounts,dc=domain,dc=tld | grep dn:
dn: cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld

[Ludwig Krispenz]

Hi,

unfortunately replication conflicts for managed entries have additional 
difficulties. The origin and managed entries reference the 
"non-conflict" entry and teh managed entry plugin prevents the deletion 
of a managed entry via ldapmodify.
To procede in cleanup you could try to remove the "mepManagedEntry" 
objectclass and "mepmanagedby" attribute and try again to delete the 
conflict entry

On 06/19/2017 06:38 PM, john.bowman--- via FreeIPA-users wrote:
>  Here is a specific example:
>
>  conflict entry:  dn:
> cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
>
>  Step 1:
>  $ ldapmodify -D "cn=directory manager" -w secret -p 389 -h ipa0.domain.tld
>  dn:
> cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld
>  changetype: modrdn
>  newrdn: cn=ipaservtemp
>  deleteoldrdn: 0
>  modifying rdn of entry
> "cn=ipaservers+nsuniqueid=e8d2f705-512111e7-9205b5bf-43202000,cn=hostgroups,cn=accounts,dc=domain,dc=tld"
>
>  Step 2:
>
>  $ ldapmodify -x -D "cn=directory manager" -w secret -p 389 -h ipa1.domain.tld
>  dn: cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld
>  changetype: modify
>  delete: cn
>  cn: ipaservers
>
>  delete: nsds5ReplConflict
>
>  Step 3:
>  $ ldapmodify -x -D "cn=directory manager" -w secret -p 389 -h ipa1.domain.tld
>  dn: cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld
>  changetype: modrdn
>  newrdn: cn=ipaservers
>  deleteoldrdn: 1
>  modifying rdn of entry
> "cn=ipaservtemp,cn=hostgroups,cn=accounts,dc=domain,dc=tld"
>
>  This produces the following error:
>  ldap_rename: Operations error (1)
>
>  When I check for the conflict its gone but on ipa1.domain.tld it gives me an error saying
> it can't find ipaservers:
>  $ ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa1.domain.tld" -D
> "cn=directory manager" -w secret -b
> cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld -s base
>  No such object (32)
>  Matched DN: cn=hostgroups,cn=accounts,dc=domain,dc=tld
>
>  But when I list all the hostgroups on that same server it does show up:
>  ldapsearch -o ldif-wrap=no -ZZ -LLLx -h "ipa1.domain.tld" -D "cn=directory
> manager" -w secret -b cn=hostgroups,cn=accounts,dc=domain,dc=tld | grep dn:
>  dn: cn=ipaservers,cn=hostgroups,cn=accounts,dc=domain,dc=tld
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org 
-- 
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric
Shander

These steps wouldn't be documented somewhere would they?  I did find this older
thread:

https://www.redhat.com/archives/freeipa-users/2016-August/msg00035.html

Something similar to those steps?

Thank you for the help very much appreciated!

[Ludwig Krispenz]

On 06/20/2017 02:31 PM, john.bowman--- via FreeIPA-users wrote:
>  These steps wouldn't be documented somewhere would they?
no, I am not aware of
>    I did find this older thread:
>
>  https://www.redhat.com/archives/freeipa-users/2016-August/msg00035.html
>
>  Something similar to those steps? 
this thread handle the removal of a managed entry
via ldapmodify, so the 
command would be similar although the reason to remove it seems to be 
different, not caused by conflicts
>  Thank you for the help very much appreciated!
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org 
-- 
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric
Shander

Yeah did not look like the same issue, but just wanted to make sure just in case.  This
gives me at least an idea on where to keep looking and I'll do a little more research
and see what else I can find on this as well before I make any changes.

Thank you very much for the help!