Dear all,

we tried to setup our first replica for our current ipa installation but failed with 

RuntimeError: Failed to start replication

Our main instance is running on Scientific Linux 7 and is already 4 years old but kept always up-to-date and served us with no problems.

We followed the steps lined out in the documentation: https://www.freeipa.org/page/V4/Replica_Setup
But we always fail at the point where the replication starts.

~# ipa-replica-install 
Run connection check to master
Connection check OK
Configuring NTP daemon (ntpd)
  [1/4]: stopping ntpd
  [2/4]: writing configuration
  [3/4]: configuring ntpd to start on boot
  [4/4]: starting ntpd
Done configuring NTP daemon (ntpd).
Configuring directory server (dirsrv). Estimated time: 30 seconds
  [1/42]: creating directory server instance
  [2/42]: enabling ldapi
  [3/42]: configure autobind for root
  [4/42]: stopping directory server
  [5/42]: updating configuration in dse.ldif
  [6/42]: starting directory server
  [7/42]: adding default schema
  [8/42]: enabling memberof plugin
  [9/42]: enabling winsync plugin
  [10/42]: configure password logging
  [11/42]: configuring replication version plugin
  [12/42]: enabling IPA enrollment plugin
  [13/42]: configuring uniqueness plugin
  [14/42]: configuring uuid plugin
  [15/42]: configuring modrdn plugin
  [16/42]: configuring DNS plugin
  [17/42]: enabling entryUSN plugin
  [18/42]: configuring lockout plugin
  [19/42]: configuring topology plugin
  [20/42]: creating indices
  [21/42]: enabling referential integrity plugin
  [22/42]: configuring certmap.conf
  [23/42]: configure new location for managed entries
  [24/42]: configure dirsrv ccache
  [25/42]: enabling SASL mapping fallback
  [26/42]: restarting directory server
  [27/42]: creating DS keytab
  [28/42]: ignore time skew for initial replication
  [29/42]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 15 seconds elapsed
[ldap://freeipa.xxx.xxx.xxx:389] reports: Update failed! Status: [Error (-2)  - LDAP error: Local error]

  [error] RuntimeError: Failed to start replication
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipapython.admintool: ERROR    Failed to start replication
ipapython.admintool: ERROR    The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

We tried to debug it a bit but did not come far. Somehow our master fails to acquire the replica for a total update (error log from dirsrv on main):

[16/Jun/2020:01:26:00.049005795 +0200] - WARN - NSMMReplicationPlugin - repl5_tot_run - Unable to acquire replica for total update, error: -2, retrying in 1 seconds.
[16/Jun/2020:01:26:01.080674785 +0200] - WARN - NSMMReplicationPlugin - repl5_tot_run - Unable to acquire replica for total update, error: -2, retrying in 2 seconds.
[16/Jun/2020:01:26:03.115527897 +0200] - WARN - NSMMReplicationPlugin - repl5_tot_run - Unable to acquire replica for total update, error: -2, retrying in 3 seconds.
[16/Jun/2020:01:26:06.137927640 +0200] - WARN - NSMMReplicationPlugin - repl5_tot_run - Unable to acquire replica for total update, error: -2, retrying in 4 seconds.
[16/Jun/2020:01:26:10.167358832 +0200] - WARN - NSMMReplicationPlugin - repl5_tot_run - Unable to acquire replica for total update, error: -2, retrying in 5 seconds.

I guess the error log on the replica is intended, since we just started to replicate it

[16/Jun/2020:01:26:00.674747749 +0200] - WARN - NSMMReplicationPlugin - repl5_inc_run - agmt="cn=meTofreeipa.i12g.informatik.tu-muenchen.de" (freeipa:389): The remote replica has a different database generation ID than the local database.  You may have to reinitialize the remote replica, or the local replica.

As we do not know if this is a bug or just a configuration issue on our side, we would appreciate any help or hints on this.
The times are synchronized btw.
To make sure we, did the the right things we tried successfully everything with a fresh installation within a VM network using CentOS 7 images.

For more details I attached the install log and the error log from our dirsrv. If you need further logs please let me know.

Some additional information from our system (our main instance):

# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: Scientific
Description:    Scientific Linux release 7.8 (Nitrogen)
Release:        7.8
Codename:       Nitrogen
# ipa --version
VERSION: 4.8.7, API_VERSION: 2.239
# yum list installed "ipa-server"
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * epel
 * sl
 * sl-fastbugs
 * sl-security
Installed Packages
ipa-server.x86_64         4.6.6-11.sl7            @sl

And from our replica system:

# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.8.2003 (Core)
Release:        7.8.2003
Codename:       Core
# ipa --version
VERSION: 4.6.6, API_VERSION: 2.231
# yum list installed ipa-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base:
 * elrepo:
 * epel:
 * extras:
 * updates:
Installed Packages
ipa-server.x86_64         4.6.6-11.el7.centos     @base

I'm just puzzled a bit by the difference in version number on the master. Could that be an issue and if so how to solve this?

Best,
Christian

-- 
Christian Mertes | PhD Student / Lab Administrator

Gagneur Lab - Computational Genomics
I12 - Department of Informa ti
Technical University of Munich
Boltzmannstr. 3, 85748 Garching, Germany

mertes@in.tum.de | https://in.tum.de/gagneurlab