Hi, I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog. I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists. I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it. Here are the logs. Any help is resolving this would be fantastic as I'm not finding much help via web searches. Thanks, David The sanitized error: Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right -------------------------------------------------------------------------------------------- Replication refresh for segment: "auth1....-to-auth2...." requested. --------------------------------------------------------------------------------------------
The sanitized logs of the re-initialization Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl). Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog. Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
Hi David,
The logs look fine after the reinit. The last lines show the changelog was successfully reinitialized (Rebuilding replication changelog RUV complete. Result 0 (Success)) But if you have a doubt then make an update on each replica and see if it's replicated to the other replica.
HTH,
Mark
On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote:
Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much help via web searches.
Thanks, David
*The sanitized error:*
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
*Santitized re-initialization*
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right
Replication refresh for segment: "auth1....-to-auth2...." requested.
*The sanitized logs of the re-initialization*
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl). Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog. Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
I was able to create a user and it replicated and was able to delete that same user from the secondary and it deleted it from the first.
I just wasn't sure if something else was in error with those logs continuing to error like that and say something was wrong with replication.
Dave
On Wednesday, April 16, 2025 at 12:04:55 PM EDT, Mark Reynolds mareynol@redhat.com wrote:
Hi David,
The logs look fine after the reinit. The last lines show the changelog was successfully reinitialized (Rebuilding replication changelog RUV complete. Result 0 (Success)) But if you have a doubt then make an update on each replica and see if it's replicated to the other replica.
HTH,
Mark
On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote:
Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much help via web searches.
Thanks, David
The sanitized error:
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right -------------------------------------------------------------------------------------------- Replication refresh for segment: "auth1....-to-auth2...." requested. --------------------------------------------------------------------------------------------
The sanitized logs of the re-initialization
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl). Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog. Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
Well it shouldn't keep initializing the changelog. You should not see it again after the message on 16/Apr/2025:10:50:31
Mark
On 4/16/25 12:19 PM, David Brown wrote:
I was able to create a user and it replicated and was able to delete that same user from the secondary and it deleted it from the first.
I just wasn't sure if something else was in error with those logs continuing to error like that and say something was wrong with replication.
Dave
On Wednesday, April 16, 2025 at 12:04:55 PM EDT, Mark Reynolds mareynol@redhat.com wrote:
Hi David,
The logs look fine after the reinit. The last lines show the changelog was successfully reinitialized (Rebuilding replication changelog RUV complete. Result 0 (Success)) But if you have a doubt then make an update on each replica and see if it's replicated to the other replica.
HTH,
Mark
On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote:
Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much help via web searches.
Thanks, David
The sanitized error:
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right
Replication refresh for segment: "auth1....-to-auth2...." requested.
The sanitized logs of the re-initialization
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl). Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog. Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
I agree; and that is usually what happens, but the fact that I still see that error every few seconds on auth2 is what concerned me and why I reached out to the group.
I'm thinking there might be a deeper issue and I'm not sure how to figure out what it is or what is causing it.
Dave
On Wednesday, April 16, 2025 at 12:37:07 PM EDT, Mark Reynolds mareynol@redhat.com wrote:
Well it shouldn't keep initializing the changelog. You should not see it again after the message on 16/Apr/2025:10:50:31
Mark
On 4/16/25 12:19 PM, David Brown wrote:
I was able to create a user and it replicated and was able to delete that same user from the secondary and it deleted it from the first.
I just wasn't sure if something else was in error with those logs continuing to error like that and say something was wrong with replication.
Dave
On Wednesday, April 16, 2025 at 12:04:55 PM EDT, Mark Reynolds mareynol@redhat.com wrote:
Hi David,
The logs look fine after the reinit. The last lines show the changelog was successfully reinitialized (Rebuilding replication changelog RUV complete. Result 0 (Success)) But if you have a doubt then make an update on each replica and see if it's replicated to the other replica.
HTH,
Mark
On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote:
Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much help via web searches.
Thanks, David
The sanitized error:
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right
Replication refresh for segment: "auth1....-to-auth2...." requested.
The sanitized logs of the re-initialization
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized. Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl). Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog. Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success) Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition. Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes... Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
Hi David,
Did you ever resolve this?
I am literally having this issue as well in a 4 node cluster. I have completely blown away 3 replicas and reinitialized them, and have also tried lowering the replication time to 300s using dsconf but it still persists, no matter what I do.
On Wed, Apr 16, 2025 at 11:52 AM David Brown via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
I agree; and that is usually what happens, but the fact that I still see that error every few seconds on auth2 is what concerned me and why I reached out to the group.
I'm thinking there might be a deeper issue and I'm not sure how to figure out what it is or what is causing it.
Dave
On Wednesday, April 16, 2025 at 12:37:07 PM EDT, Mark Reynolds < mareynol@redhat.com> wrote:
Well it shouldn't keep initializing the changelog. You should not see it again after the message on 16/Apr/2025:10:50:31
Mark
On 4/16/25 12:19 PM, David Brown wrote:
I was able to create a user and it replicated and was able to delete
that same user from the secondary and it deleted it from the first.
I just wasn't sure if something else was in error with those logs
continuing to error like that and say something was wrong with replication.
Dave
On Wednesday, April 16, 2025 at 12:04:55 PM EDT, Mark Reynolds <
mareynol@redhat.com> wrote:
Hi David,
The logs look fine after the reinit. The last lines show the changelog
was successfully reinitialized (Rebuilding replication changelog RUV complete. Result 0 (Success)) But if you have a doubt then make an update on each replica and see if it's replicated to the other replica.
HTH,
Mark
On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote:
Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today
that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has
fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication
and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much
help via web searches.
Thanks, David
The sanitized error:
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677
-0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized.
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395
-0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312
-0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right
Replication refresh for segment: "auth1....-to-auth2...." requested.
The sanitized logs of the re-initialization
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886
-0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380
-0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600
-0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813
-0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized.
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190
-0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030
-0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677
-0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770
-0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951
-0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375
-0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267
-0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818
-0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319
-0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820
-0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl).
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646
-0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec)
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417
-0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900
-0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353
-0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog.
Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file.
This could affect replication with replica's consumers in which case the consumers should be reinitialized.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463
-0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319
-0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720
-0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418
-0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194
-0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262
-0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
-- Identity Management Development Team
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
freeipa-users@lists.fedorahosted.org
-
David Brown -
Mark Reynolds -
Russell Jones