Re: Can't Add Replica: The changelog directory CLDB already exists and is not empty - FreeIPA-users - Fedora mailing-lists

1 Jun 2021


      This is an old thread but I’m running into this issue and was wondering if there was ever a resolution to this.
Tldr -
My master failed and was not able to start up due to the dse.ldif being a zero byte file and the .bak file was unusable as well.  Ended up using the startOK file and that got my IPA master back up.  I didn’t find out till a week or so later that my replication has stopped working and I’ve been trying to resolve this ever since.
The error I’m getting when trying to set up a new replica is the error in the subject.  These are the last couple entries in the journal logs for the dirsrv service :
May 20 16:11:40 ns-slapd[5273]: [20/May/2021:16:11:40.900845676 +0000] - NOTICE - bdb_start - Detected Disorderly Shutdown last time Directory Server was running, recovering database.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.103929069 +0000] - ERR - attrcrypt_unwrap_key - Failed to unwrap key for cipher AES
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.106523128 +0000] - ERR - attrcrypt_cipher_init - Symmetric key failed to unwrap with the private key; Cert might have been renewed since the key is wrapped.  To recover the encrypted contents, keep the wrapped symmetric key value.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.281157478 +0000] - ERR - attrcrypt_unwrap_key - Failed to unwrap key for cipher 3DES
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.284236656 +0000] - ERR - attrcrypt_cipher_init - Symmetric key failed to unwrap with the private key; Cert might have been renewed since the key is wrapped.  To recover the encrypted contents, keep the wrapped symmetric key value.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.287235192 +0000] - ERR - attrcrypt_init - All prepared ciphers are not available. Please disable attribute encryption.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.464658571 +0000] - ERR - attrcrypt_unwrap_key - Failed to unwrap key for cipher AES
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.468260771 +0000] - ERR - attrcrypt_cipher_init - Symmetric key failed to unwrap with the private key; Cert might have been renewed since the key is wrapped.  To recover the encrypted contents, keep the wrapped symmetric key value.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.644832465 +0000] - ERR - attrcrypt_unwrap_key - Failed to unwrap key for cipher 3DES
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.647838123 +0000] - ERR - attrcrypt_cipher_init - Symmetric key failed to unwrap with the private key; Cert might have been renewed since the key is wrapped.  To recover the encrypted contents, keep the wrapped symmetric key value.
May 20 16:11:42 ns-slapd[5273]: [20/May/2021:16:11:42.650519798 +0000] - ERR - attrcrypt_init - All prepared ciphers are not available. Please disable attribute encryption.
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.015851937 +0000] - ERR - schema-compat-plugin - scheduled schema-compat-plugin tree scan in about 5 seconds after the server startup!
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.054457416 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.056902182 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.059621578 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.061834684 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.063891013 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.066217133 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.068870945 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.071006284 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.073207989 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.076186848 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.078837082 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.081064756 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.083418248 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.085693933 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.088486548 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.090954337 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.105391221 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ad,cn=etc,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.109923564 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.111808229 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=example,dc=net does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.199628452 +0000] - WARN - NSACLPlugin - acl_parse - The ACL target cn=automember rebuild membership,cn=tasks,cn=config does not exist
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.207869328 +0000] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=example,dc=net--no CoS Templates found, which should be added before the CoS Definition.
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.251700304 +0000] - INFO - slapd_daemon - slapd started.  Listening on All Interfaces port 389 for LDAP requests
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.254651872 +0000] - INFO - slapd_daemon - Listening on All Interfaces port 636 for LDAPS requests
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.256778704 +0000] - INFO - slapd_daemon - Listening on /var/run/slapd-EXAMPLE-NET.socket for LDAPI requests
May 20 16:11:43 systemd[1]: Started 389 Directory Server EXAMPLE-NET..
May 20 16:11:43 ns-slapd[5273]: [20/May/2021:16:11:43.310441141 +0000] - ERR - schema-compat-plugin - schema-compat-plugin tree scan will start in about 5 seconds!
May 20 16:11:48 ns-slapd[5273]: [20/May/2021:16:11:48.503046676 +0000] - ERR - schema-compat-plugin - warning: no entries set up under cn=computers, cn=compat,dc=example,dc=net
May 20 16:11:48 ns-slapd[5273]: [20/May/2021:16:11:48.514741500 +0000] - ERR - schema-compat-plugin - Finished plugin initialization.
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.319674451 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.325071163 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.329293579 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.333178665 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.336932011 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.341244859 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.345131920 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.349357371 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:03 ns-slapd[5273]: [20/May/2021:16:12:03.353178446 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 20 16:12:51 ns-slapd[5273]: [20/May/2021:16:12:51.527767324 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=77 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:12:52 ns-slapd[5273]: [20/May/2021:16:12:52.283753249 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=78 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:12:52 ns-slapd[5273]: [20/May/2021:16:12:52.390379930 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=79 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:12:52 ns-slapd[5273]: [20/May/2021:16:12:52.957417497 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=80 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:12:53 ns-slapd[5273]: [20/May/2021:16:12:53.283781064 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=81 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:12:55 ns-slapd[5273]: [20/May/2021:16:12:55.479234600 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=82 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:15:51 ns-slapd[5273]: [20/May/2021:16:15:51.868329611 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=212 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:16:24 ns-slapd[5273]: [20/May/2021:16:16:24.216095880 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=233 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 20 16:16:27 ns-slapd[5273]: [20/May/2021:16:16:27.408505127 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=240 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 25 20:45:37 ns-slapd[5273]: [25/May/2021:20:45:37.356300061 +0000] - ERR - NSMMReplicationPlugin - multimaster_extop_StartNSDS50ReplicationRequest - conn=111801 op=5 replica="unknown": Unable to acquire replica: error: no such replica
May 26 04:20:37 ns-slapd[5273]: [26/May/2021:04:20:37.246445897 +0000] - ERR - ipa-topology-plugin - ipa_topo_agreement_dn: no replica found
May 26 04:20:37 ns-slapd[5273]: [26/May/2021:04:20:37.249257028 +0000] - ERR - ipa-topology-plugin - ipa_topo_agmt_del: cn=master.example.net-to-replica001.example.net
May 26 04:20:39 ns-slapd[5273]: [26/May/2021:04:20:39.266434467 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:41 ns-slapd[5273]: [26/May/2021:04:20:41.272692883 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:43 ns-slapd[5273]: [26/May/2021:04:20:43.333985925 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:43 ns-slapd[5273]: [26/May/2021:04:20:43.337030838 +0000] - ERR - NSMMReplicationPlugin - CleanAllRUV Task (rid 5): Could not find replica from dn(dc=example,dc=net)
May 26 04:20:45 ns-slapd[5273]: [26/May/2021:04:20:45.342517080 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:47 ns-slapd[5273]: [26/May/2021:04:20:47.348898719 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:49 ns-slapd[5273]: [26/May/2021:04:20:49.355780507 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:49 ns-slapd[5273]: [26/May/2021:04:20:49.358756218 +0000] - ERR - NSMMReplicationPlugin - CleanAllRUV Task (rid 5): Task failed...(-1)
May 26 04:20:51 ns-slapd[5273]: [26/May/2021:04:20:51.364127080 +0000] - WARN - modify_internal_entry - Can't modify task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'; No such object (32)
May 26 04:20:51 ns-slapd[5273]: [26/May/2021:04:20:51.406580664 +0000] - WARN - get_internal_entry - Can't find task entry 'cn=clean 5,cn=cleanallruv,cn=tasks,cn=config'
May 26 04:20:51 ns-slapd[5273]: [26/May/2021:04:20:51.412684547 +0000] - ERR - ipa-topology-plugin - ipa_topo_util_cleanruv: failed to create cleanalltuv task
May 28 00:08:17 ns-slapd[5273]: [28/May/2021:00:08:17.669467056 +0000] - ERR - log_ber_too_big_error - conn=173723 fd=156 Incoming BER Element may be misformed. This may indicate an attempt to use TLS on a plaintext port, IE ldaps://localhost:389. Check your client LDAP_URI settings.
May 28 01:06:22 ns-slapd[5273]: [28/May/2021:01:06:22.406718855 +0000] - ERR - log_ber_too_big_error - conn=175016 fd=158 Incoming BER Element was 24019198018235050 bytes, max allowable is 209715200 bytes. Change the nsslapd-maxbersize attribute in cn=config to increase.
May 28 15:50:43 ns-slapd[5273]: [28/May/2021:15:50:43.082273849 +0000] - ERR - log_ber_too_big_error - conn=195035 fd=289 Incoming BER Element may be misformed. This may indicate an attempt to use TLS on a plaintext port, IE ldaps://localhost:389. Check your client LDAP_URI settings.
May 28 15:50:43 ns-slapd[5273]: [28/May/2021:15:50:43.097752625 +0000] - ERR - log_ber_too_big_error - conn=195036 fd=289 Incoming BER Element may be misformed. This may indicate an attempt to use TLS on a plaintext port, IE ldaps://localhost:389. Check your client LDAP_URI settings.
May 29 12:43:13 ns-slapd[5273]: [29/May/2021:12:43:13.872403558 +0000] - ERR - log_ber_too_big_error - conn=222810 fd=357 Incoming BER Element may be misformed. This may indicate an attempt to use TLS on a plaintext port, IE ldaps://localhost:389. Check your client LDAP_URI settings.
May 29 17:26:04 ns-slapd[5273]: [29/May/2021:17:26:04.858100977 +0000] - ERR - log_ber_too_big_error - conn=229005 fd=322 Incoming BER Element may be misformed. This may indicate an attempt to use TLS on a plaintext port, IE ldaps://localhost:389. Check your client LDAP_URI settings.
May 31 11:05:27 ns-slapd[5273]: [31/May/2021:11:05:27.982685756 +0000] - ERR - connection_read_operation - conn=283764 received a non-LDAP message (tag 0x47, expected 0x30)
May 31 11:05:31 ns-slapd[5273]: [31/May/2021:11:05:31.522716719 +0000] - ERR - connection_read_operation - conn=283766 received a non-LDAP message (tag 0x47, expected 0x30)
May 31 11:31:27 ns-slapd[5273]: [31/May/2021:11:31:27.029834838 +0000] - ERR - connection_read_operation - conn=284343 received a non-LDAP message (tag 0x47, expected 0x30)
May 31 11:31:27 ns-slapd[5273]: [31/May/2021:11:31:27.520938917 +0000] - ERR - connection_read_operation - conn=284344 received a non-LDAP message (tag 0x47, expected 0x30)
I changed up the host info in the log output but otherwise log is still the same.
Right now, as it is, the master works, the existing replicas are working but no new changes are getting pushed out.  I would like to NOT rebuild the entire IPA infrastructure if I can avoid it to get replication back up and running so any help would be greatly appreciated.
Thank you.
Sinh Lam