replica install fails: CA_UNREACHABLE

[lejeczek]

hi

I'm trying to install replica, process fails:
..
   [3/5]: creating anonymous principal
   [4/5]: starting the KDC
   [5/5]: configuring KDC to start on boot
Done configuring Kerberos KDC (krb5kdc).
Configuring kadmin
   [1/2]: starting kadmin
   [2/2]: configuring kadmin to start on boot
Done configuring kadmin.
Configuring directory server (dirsrv)
   [1/3]: configuring TLS for DS instance
   [error] RuntimeError: Certificate issuance failed 
(CA_UNREACHABLE)
Your system may be partly configured.
..
-- end

and in intall log file:
..
2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
/etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
/etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
2018-01-06T13:50:29Z DEBUG Process finished, return code=0
2018-01-06T13:50:29Z DEBUG stdout=
2018-01-06T13:50:29Z DEBUG stderr=
2018-01-06T13:50:30Z DEBUG certmonger request is in state 
dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
2018-01-06T13:50:35Z DEBUG certmonger request is in state 
dbus.String(u'CA_UNREACHABLE', variant_level=1)
2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 504, in start_creation
     run_step(full_msg, method)
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 494, in run_step
     method()
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
line 824, in __enable_ssl
     post_command=cmd)
   File 
"/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", 
line 317, in request_and_wait_for_cert
     raise RuntimeError("Certificate issuance failed 
({})".format(state))
RuntimeError: Certificate issuance failed (CA_UNREACHABLE)

2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: 
Certificate issuance failed (CA_UNREACHABLE)
2018-01-06T13:50:35Z DEBUG   File 
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", 
line 172, in execute
     return_value = self.run()
   File 
"/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
line 333, in run
     cfgr.run()
   File "/usr/lib/python2.7/site-
...
-- end

Would this be that new candidate's problem or some 
communication issues with existing server? Client installed 
(kind of)okey though.

[Florence Blanc-Renaud]

On 01/06/2018 08:54 PM, lejeczek via FreeIPA-users wrote:
> hi
>
> I'm trying to install replica, process fails:
> ..
>    [3/5]: creating anonymous principal
>    [4/5]: starting the KDC
>    [5/5]: configuring KDC to start on boot
> Done configuring Kerberos KDC (krb5kdc).
> Configuring kadmin
>    [1/2]: starting kadmin
>    [2/2]: configuring kadmin to start on boot
> Done configuring kadmin.
> Configuring directory server (dirsrv)
>    [1/3]: configuring TLS for DS instance
>    [error] RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
> Your system may be partly configured.
> ..
> -- end
>
> and in intall log file:
> ..
> 2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
> PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
> 2018-01-06T13:50:29Z DEBUG Process finished, return code=0
> 2018-01-06T13:50:29Z DEBUG stdout=
> 2018-01-06T13:50:29Z DEBUG stderr=
> 2018-01-06T13:50:30Z DEBUG certmonger request is in state 
> dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG certmonger request is in state 
> dbus.String(u'CA_UNREACHABLE', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
>    File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 504, in start_creation
>      run_step(full_msg, method)
>    File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 494, in run_step
>      method()
>    File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 
> 824, in __enable_ssl
>      post_command=cmd)
>    File "/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", 
> line 317, in request_and_wait_for_cert
>      raise RuntimeError("Certificate issuance failed ({})".format(state))
> RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
>
> 2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: Certificate issuance 
> failed (CA_UNREACHABLE)
> 2018-01-06T13:50:35Z DEBUG   File 
> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in 
> execute
>      return_value = self.run()
>    File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
> line 333, in run
>      cfgr.run()
>    File "/usr/lib/python2.7/site-
> ...
> -- end
>
> Would this be that new candidate's problem or some communication issues 
> with existing server? Client installed (kind of)okey though.
> _______________________________________________
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

Hi,

the replica installer is communicating with the local certmonger daemon 
to request SSL certificates. Then certmonger connects to the IPA master 
(httpd process), and in turn IPA master server communicates with Dogtag 
to request the certificate.

As you can see, there are a lot of processes involved, and the issue 
could come from communication issues between all of them. We need to 
identify which step is failing.

Can you check:
- the output of getcert list on the client? It may contain a more 
detailed message for the certificate issuance failure
- if tomcat is running on the master? systemctl status 
pki-tomcatd@pki-tomcat
- if the client managed to contact IPA master? Look for a line with 
cert_request on the master's log /var/log/httpd/error_log, and for 
possible error messages related. If the line is present, the client 
successfully sent its cert request, meaning that the communication was 
properly established.
- if dogtag received the certificate request? IPA master is using 
/etc/ipa/ca.crt and /var/lib/ipa/ra-agent.{key|pem} to authenticate to 
Dogtag. The authentication logs in /var/log/pki/pki-tomcat/ca/debug 
should display something like:

[date][ajp-bio-127.0.0.1-8009-exec-1]: PKIRealm: Authenticating 
certificate chain:
[date][ajp-bio-127.0.0.1-8009-exec-1]: PKIRealm.getAuditUserfromCert: 
certUID=CN=IPA RA, O=DOMAIN.IPA.COM
[date][ajp-bio-127.0.0.1-8009-exec-1]: PKIRealm:   CN=IPA RA, 
O=DOMAIN.IPA.COM
[date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: started
[date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: Retrieving client 
certificate
[date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: Got client 
certificate

and the cert request:
[date][ajp-bio-127.0.0.1-8009-exec-4]: EnrollProfile: createRequests: begins
[date][ajp-bio-127.0.0.1-8009-exec-4]: Start parsePKCS10(): -----BEGIN 
CERTIFICATE REQUEST-----

The most common issues are pki-tomcatd not started because of the 
certificate 'subsystemCert cert-pki-ca' that expired, or communication 
issues between IPA server and Dogtag (the cert in 
/var/lib/ipa/ra-agent.{key|pem} is expired).

HTH,
Flo

[lejeczek]

On 08/01/18 09:36, Florence Blanc-Renaud wrote:
> On 01/06/2018 08:54 PM, lejeczek via FreeIPA-users wrote:
> > hi
> >
> > I'm trying to install replica, process fails:
> > ..
> >    [3/5]: creating anonymous principal
> >    [4/5]: starting the KDC
> >    [5/5]: configuring KDC to start on boot
> > Done configuring Kerberos KDC (krb5kdc).
> > Configuring kadmin
> >    [1/2]: starting kadmin
> >    [2/2]: configuring kadmin to start on boot
> > Done configuring kadmin.
> > Configuring directory server (dirsrv)
> >    [1/3]: configuring TLS for DS instance
> >    [error] RuntimeError: Certificate issuance failed 
> > (CA_UNREACHABLE)
> > Your system may be partly configured.
> > ..
> > -- end
> >
> > and in intall log file:
> > ..
> > 2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
> > /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
> > PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
> > /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
> >
> > 2018-01-06T13:50:29Z DEBUG Process finished, return code=0
> > 2018-01-06T13:50:29Z DEBUG stdout=
> > 2018-01-06T13:50:29Z DEBUG stderr=
> > 2018-01-06T13:50:30Z DEBUG certmonger request is in state 
> > dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
> > 2018-01-06T13:50:35Z DEBUG certmonger request is in state 
> > dbus.String(u'CA_UNREACHABLE', variant_level=1)
> > 2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
> >    File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> > line 504, in start_creation
> >      run_step(full_msg, method)
> >    File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> > line 494, in run_step
> >      method()
> >    File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
> > line 824, in __enable_ssl
> >      post_command=cmd)
> >    File 
> > "/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", 
> > line 317, in request_and_wait_for_cert
> >      raise RuntimeError("Certificate issuance failed 
> > ({})".format(state))
> > RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
> >
> > 2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: 
> > Certificate issuance failed (CA_UNREACHABLE)
> > 2018-01-06T13:50:35Z DEBUG   File 
> > "/usr/lib/python2.7/site-packages/ipapython/admintool.py", 
> > line 172, in execute
> >      return_value = self.run()
> >    File 
> > "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
> > line 333, in run
> >      cfgr.run()
> >    File "/usr/lib/python2.7/site-
> > ...
> > -- end
> >
> > Would this be that new candidate's problem or some 
> > communication issues with existing server? Client 
> > installed (kind of)okey though.
> > _______________________________________________
> > FreeIPA-users mailing list -- 
> > [e-mail]
> > To unsubscribe send an email to 
> > [e-mail]
>
> Hi,
>
> the replica installer is communicating with the local 
> certmonger daemon to request SSL certificates. Then 
> certmonger connects to the IPA master (httpd process), and 
> in turn IPA master server communicates with Dogtag to 
> request the certificate.
>
> As you can see, there are a lot of processes involved, and 
> the issue could come from communication issues between all 
> of them. We need to identify which step is failing.
>
> Can you check:
> the output of getcert list on the client? It may contain
>
> a more detailed message for the certificate issuance failure

After a client successful installation, on a client:
# getcert list
Number of certificates and requests being tracked: 0.

The same on the server shows long list of:
Number of certificates and requests being tracked: 9.
...
> if tomcat is running on the master? systemctl status
>
> pki-tomcatd@pki-tomcat

Is running
> if the client managed to contact IPA master? Look for a
>
> line with cert_request on the master's log 
> /var/log/httpd/error_log, and for possible error messages 
> related. If the line is present, the client successfully 
> sent its cert request, meaning that the communication was 
> properly established.

Just after a client installation, now 
--uninstall(successful), on the server in httpd/error_log:
...
[Tue Jan 09 12:03:45.109806 2018] [auth_gssapi:error] [pid 
34824] [client 10.5.10.37:46016] NO AUTH DATA Client did not 
send any authentication headers, referer: 
https://swir.priv.xx.xx.priv.xx.xx.x/ipa/xml
[Tue Jan 09 12:03:45.771504 2018] [:error] [pid 8044] ipa: 
INFO: [xmlserver] 
host/[e-mail].x: 
host_disable(u'lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x', 
version=u'2.51'): ACIError

And I when I re-install a client, on the server in 
httpd/error_log:
...
[Tue Jan 09 12:05:37.515855 2018] [auth_gssapi:error] [pid 
8048] [client 10.5.10.37:46108] NO AUTH DATA Client did not 
send any authentication headers, referer: 
https://swir.priv.xx.xx.priv.xx.xx.x/ipa/xml
[Tue Jan 09 12:05:37.579441 2018] [:error] [pid 8043] ipa: 
INFO: Host entry for lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x 
already exists, joining may fail on the client side if not 
forced
[Tue Jan 09 12:05:37.628158 2018] [:error] [pid 8043] ipa: 
INFO: [xmlserver] [e-mail].x: 
join(u'lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x', 
nshardwareplatform=u'x86_64', 
nsosversion=u'3.10.0-693.11.6.el7.x86_64', version=u'2.51'): 
SUCCESS

And a client's end:
...
Joining realm failed: Host is already joined.

Use --force-join option to override the host entry on the 
server and force client enrollment.
Installation failed. Rolling back changes.
Unconfigured automount client failed: Command 
'ipa-client-automount --uninstall --debug' returned non-zero 
exit status 1
...

So, --force-join now:
...
The ipa-client-install command was successful
...

Servers's end in httpd/error_log:
...
[Tue Jan 09 12:09:36.546759 2018] [auth_gssapi:error] [pid 
8046] [client 10.5.10.37:46266] NO AUTH DATA Client did not 
send any authentication headers, referer: 
https://swir.priv.xx.xx.priv.xx.xx.x/ipa/xml
[Tue Jan 09 12:09:36.602576 2018] [:error] [pid 8044] ipa: 
INFO: Host entry for lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x 
already exists, joining may fail on the client side if not 
forced
[Tue Jan 09 12:09:36.656072 2018] [:error] [pid 8044] ipa: 
INFO: [xmlserver] [e-mail].x: 
join(u'lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x', 
nshardwareplatform=u'x86_64', 
nsosversion=u'3.10.0-693.11.6.el7.x86_64', version=u'2.51'): 
SUCCESS
[Tue Jan 09 12:09:39.203290 2018] [:error] [pid 8043] ipa: 
INFO: [jsonserver_kerb] 
host/[e-mail].x: 
ping(): SUCCESS
[Tue Jan 09 12:09:39.244366 2018] [:error] [pid 8044] ipa: 
INFO: [jsonserver_kerb] 
host/[e-mail].x: 
ca_is_enabled(version=u'2.107'): SUCCESS
[Tue Jan 09 12:09:40.430598 2018] [:error] [pid 8043] ipa: 
INFO: [jsonserver_kerb] 
host/[e-mail].x: 
host_mod(u'lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x', 
ipasshpubkey=(u'ssh-rsa 
AAAAB3NzaC1yc2EAAAADAQABAAABAQDTFpZYnI+r+wM4wYPQgEIu21k6UKpkiPhgYhTlCn29GcyXNoIipnxyrV4LRVw5VULWMsuZQcxK7DB897VNkR/wm0aeOtyWYDGEqwVclhVy2yWPwnD4scltCCS2ehv20yeO0V0Uj95EMGdFOfUM3FGpFG9L/xEXlSVdC6BP/oi3DxCcGydGas6SCwmzy8av10hB+jwQXgnLeHGIo2bPZZuWwxjH2rifaoZjsOH4I/EfRkYq3Q2JsBBGAWhNhuOgxBeUIVQOvBvrMf4JeRGFHNQNsvhElxijy0Bxi2uGADisKKWzeW5vnslTNtOtFu9AwoJg52kV2HM5Wuo+crp4+zF/', 
u'ecdsa-sha2-nistp256 
AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFN0Gfh8oGhmjnhOqhEdK7xNPlWkoimaWdtiBwJo2+QBEQ0s6Qvjc4WPA+zCF8ELA/Lg3RHbsSjRTQc3N3jRxDA=', 
u'ssh-ed25519 
AAAAC3NzaC1lZDI1NTE5AAAAIPs2DCmihX985GR3k/Q4cHyeBVqmM72pB5zNo5fHehfG'), 
updatedns=False, version=u'2.26'): EmptyModlist

On a client again:
# getcert list
Number of certificates and requests being tracked: 0.

So there is something not right, right? Client --uninstall 
leaves stuff behind?

--- Now replica:
# ipa-replica-install
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipa.ipapython.install.cli.install_tool(CompatServerReplicaInstall): 
ERROR    A replication agreement for this host already 
exists. It needs to be removed.
Run this command:
     %% ipa-replica-manage del 
lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x --force
ipa.ipapython.install.cli.install_tool(CompatServerReplicaInstall): 
ERROR
...

But there is no such agreement the severs sais, only one 
agreement to itself. And no matter what I do I cannot get 
pass this, -replica insists and fails, unless I do on the 
client:
# ipa-server-install --uninstall
# yum remove -y `rpm -qa ipa* 389*` pki-base krb5-pkinit 
krb5-server krb5-workstation ipa-python certmonger
# yum install -y ipa-server bind bind-dyndb-ldap ipa-server-dns

Then anew client-install, then replica-install:
# ipa-replica-install
ipa         : ERROR    Reverse DNS resolution of address 
10.5.10.37 (lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x) failed. 
Clients may not function properly. Please check your DNS 
setup. (Note that this check queries IPA DNS directly and 
ignores /etc/hosts.)
...

Finally the moment -replica fails on the server in 
httpd/error_log:
...
[Tue Jan 09 12:39:15.256308 2018] [auth_gssapi:error] [pid 
8050] [client 10.5.10.37:48282] NO AUTH DATA Client did not 
send any authentication headers, referer: 
https://swir.priv.xx.xx.priv.xx.xx.x/ipa/xml
[Tue Jan 09 12:39:15.329299 2018] [:error] [pid 8044] ipa: 
INFO: [xmlserver] 
host/[e-mail].x: 
cert_request(u'MIIEejCCA2ICAQAwaDErMCkGA1UEChMiUFJJVkFURS5DQ05SLkNFQi5QUklWQVRFLkNBTS5BQy5VSzE5MDcGA1UEAxMwbHhjLWlwYTEtc3dpci5wcml2YXRlLmNjbnIuY2ViLnByaXZhdGUuY2FtLmFjLnVrMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3FdjuzlDp8Z6F5cZf/E58yATNkiFh3IwJeoZpUNmyTUzdNpUTL7rLeiEGXRsGZPBV82R78AJK0aJNidU/2cbmiOQXB8msol9SK7D0COVtaTLhBXnurvt7Mya/QT2rlvrRtklaQ3zjO9h+ogzYwKtMVwy843F+rTWSAHuuSpFwYR0u6n716q3P4QUnD+M+xWy74ir1EdotwSXbwfVzx/egOzSB3hm0O4Xjp82A8DwscV3rmLi80Q0R8LkI3s7tBbc2sCcyL/zfflc7potwAL/DYw+j8kLrWmVSgl2Qk6SlxxtmjNRd+KZumdVuy8cFHIyuNyvTRitIjhZdMSTNgsmpQIDAQABoIIByzAlBgkqhkiG9w0BCRQxGB4WAFMAZQByAHYAZQByAC0AQwBlAHIAdDCCAaAGCSqGSIb3DQEJDjGCAZEwggGNMIIBJQYDVR0RAQEABIIBGTCCARWCMGx4Yy1pcGExLXN3aXIucHJpdmF0ZS5jY25yLmNlYi5wcml2YXRlLmNhbS5hYy51a6BoBgorBgEEAYI3FAIDoFoMWGxkYXAvbHhjLWlwYTEtc3dpci5wcml2YXRlLmNjbnIuY2ViLnByaXZhdGUuY2FtLmFjLnVrQFBSSVZBVEUuQ0NOUi5DRUIuUFJJVkFURS5DQU0uQUMuVUugdwYGKwYBBQICoG0wa6AkGyJQUklWQVRFLkNDTlIuQ0VCLlBSSVZBVEUuQ0FNLkFDLlVLoUMwQaADAgEBoTowOBsEbGRhcBswbHhjLWlwYTEtc3dpci5wcml2YXRlLmNjbnIuY2ViLnByaXZhdGUuY2FtLmFjLnVrMAwGA1UdEwEB/wQCMAAwIAYDVR0OAQEABBYEFFKuq49asd1Mcq+MlTIawY4s0BwVMDIGCSsGAQQBgjcUAgEBAAQiHiAAYwBhAEkAUABBAHMAZQByAHYAaQBjAGUAQwBlAHIAdDANBgkqhkiG9w0BAQsFAAOCAQEABZeuyLxPx8ASsSMOQToYZKKlmI6NUopd9kFfx4GLbtR788qLCx2nIbwTpWTfpSBEDZRBo9X/7ejUZwGmjotYriM5O/Kq3o+onxj3kvv45vWd4O8ZcsoQseROeuTW0b/X3RT7riaKX6RD1J4tJrFi3g4Hz3nF0odWfLGr00TTYqeMUaCn4iwaKRjFq2bV0VZeANvF3S/rNzUfYc+DQp1qAZpdgLH2mDcYN4c8r7V+cxLblku8aOjnsmExXZRDhAOflPOz9ZtaflEhe+UwENP+nqRpjezy5FWzFwQ4P0epYnFpEeA7cagqavuKQelMge8nQc4YQtZcYO9TnbxDTAEGhw==', 
profile_id=u'caIPAserviceCert', 
principal=u'ldap/[e-mail].x', 
add=True, version=u'2.51'): NetworkError

Nothing happens in that moment in 
/var/log/pki/pki-tomcat/ca/debug

Before I thought... maybe network, but now, all above I do 
is in a lxc container on that IPA server locally.
Is this IPA server somehow functional but not, 
mis-configured? (Installation of the server seemingly went okey)
I've done a few IPA setups, most of the time problem-free, 
sometimes minor problems, this is first time I'm at a loss.
> if dogtag received the certificate request? IPA master
>
> is using /etc/ipa/ca.crt and 
> /var/lib/ipa/ra-agent.{key|pem} to authenticate to Dogtag. 
> The authentication logs in 
> /var/log/pki/pki-tomcat/ca/debug should display something 
> like:
>
> [date][ajp-bio-127.0.0.1-8009-exec-1]: PKIRealm: 
> Authenticating certificate chain:
> [date][ajp-bio-127.0.0.1-8009-exec-1]: 
> PKIRealm.getAuditUserfromCert: certUID=CN=IPA RA, 
> O=DOMAIN.IPA.COM
> [date][ajp-bio-127.0.0.1-8009-exec-1]: PKIRealm:   CN=IPA 
> RA, O=DOMAIN.IPA.COM
> [date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: 
> started
> [date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: 
> Retrieving client certificate
> [date][ajp-bio-127.0.0.1-8009-exec-1]: CertUserDBAuth: Got 
> client certificate
>
> and the cert request:
> [date][ajp-bio-127.0.0.1-8009-exec-4]: EnrollProfile: 
> createRequests: begins
> [date][ajp-bio-127.0.0.1-8009-exec-4]: Start 
> parsePKCS10(): -----BEGIN CERTIFICATE REQUEST-----
>
> The most common issues are pki-tomcatd not started because 
> of the certificate 'subsystemCert cert-pki-ca' that 
> expired, or communication issues between IPA server and 
> Dogtag (the cert in /var/lib/ipa/ra-agent.{key|pem} is 
> expired).
>
> HTH,
> Flo

[lejeczek]

On 06/01/18 19:54, lejeczek via FreeIPA-users wrote:
> hi
>
> I'm trying to install replica, process fails:
> ..
>   [3/5]: creating anonymous principal
>   [4/5]: starting the KDC
>   [5/5]: configuring KDC to start on boot
> Done configuring Kerberos KDC (krb5kdc).
> Configuring kadmin
>   [1/2]: starting kadmin
>   [2/2]: configuring kadmin to start on boot
> Done configuring kadmin.
> Configuring directory server (dirsrv)
>   [1/3]: configuring TLS for DS instance
>   [error] RuntimeError: Certificate issuance failed 
> (CA_UNREACHABLE)
> Your system may be partly configured.
> ..
> -- end

And if -replica failed as above then this also fails on the 
same candidate client:

# ipa-server-install --uninstall

This is a NON REVERSIBLE operation and will delete all data 
and configuration!
It is highly recommended to take a backup of existing data 
and configuration using ipa-backup utility before proceeding.

Are you sure you want to continue with the uninstall 
procedure? [no]: yes
ipa.ipapython.install.cli.uninstall_tool(CompatServerMasterInstall): 
ERROR    Server removal aborted:

Replication topology in suffix 'domain' is disconnected:
Topology does not allow server swir.priv.xx.xx.priv.xx.xx.x 
to replicate with servers:
     lxc-ipa1-swir.priv.xx.xx.priv.xx.xx.x.
ipa.ipapython.install.cli.uninstall_tool(CompatServerMasterInstall): 
ERROR    The ipa-server-install command failed. See 
/var/log/ipaserver-uninstall.log for more information
> and in intall log file:
> ..
> 2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
> PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
> 2018-01-06T13:50:29Z DEBUG Process finished, return code=0
> 2018-01-06T13:50:29Z DEBUG stdout=
> 2018-01-06T13:50:29Z DEBUG stderr=
> 2018-01-06T13:50:30Z DEBUG certmonger request is in state 
> dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG certmonger request is in state 
> dbus.String(u'CA_UNREACHABLE', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 504, in start_creation
>     run_step(full_msg, method)
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 494, in run_step
>     method()
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
> line 824, in __enable_ssl
>     post_command=cmd)
>   File 
> "/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", 
> line 317, in request_and_wait_for_cert
>     raise RuntimeError("Certificate issuance failed 
> ({})".format(state))
> RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
>
> 2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: 
> Certificate issuance failed (CA_UNREACHABLE)
> 2018-01-06T13:50:35Z DEBUG   File 
> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", 
> line 172, in execute
>     return_value = self.run()
>   File 
> "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
> line 333, in run
>     cfgr.run()
>   File "/usr/lib/python2.7/site-
> ...
> -- end
>
> Would this be that new candidate's problem or some 
> communication issues with existing server? Client 
> installed (kind of)okey though.
> _______________________________________________
> FreeIPA-users mailing list -- 
> [e-mail]
> To unsubscribe send an email to 
> [e-mail]

[Charles Hedrick]

I also had issues installing a replica under 7.4. Here are my notes. krb4 is the new replica, krb1 and 2 the existing ones.

However a few things set up on krb4 didn't replicate to the krb1 and krb2. There were enough issues that I did a full comparison of dumps from krb1 and krb4. Use "/sbin/ipa-backup --online --data" to dump the data, untar the tar file, and look at CS-RUTGERS-EDU-userRoot.ldif. I used ldifsort.pl and ldifdiff.pl to compare the versions from krb1 and krb4, together with an awk script that normalizes entries. So I did a full comparison of ldif files (from a backup). I found three things:

dn: cn=krb4.cs.rutgers.eduhttp://krb4.cs.rutgers.edu,cn=masters,cn=ipa,cn=etc,dc=cs,dc=rutgers,dc=edu
changetype:modify
add:objectclass
objectClass: nsContainer
objectClass: ipaReplTopoManagedServer
objectClass: ipaConfigObject
objectClass: ipaSupportedDomainLevelConfig
-
add:ipaReplTopoManagedSuffix
ipaReplTopoManagedSuffix: dc=cs,dc=rutgers,dc=edu
-
add:ipaMinDomainLevel
ipaMinDomainLevel: 0
-
add:ipaMaxDomainLevel
ipaMaxDomainLevel: 1

dn: cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,dc=cs,dc=rutgers,dc=edu
changetype:modify
add:memberprincipal
memberPrincipal: HTTP/[e-mail]

dn: cn=ipa-ldap-delegation-targets,cn=s4u2proxy,cn=etc,dc=cs,dc=rutgers,dc=edu
changetype:modify
add:memberprincipal
memberPrincipal: ldap/[e-mail]

Perhaps a cleaner solution would have been to reinit the other two from krb4, but I wasn't sure that was safe.
In addition to this, "ipa topologysegment-find domain" showed that the link to krb4 was one-way. I later realized that the repliction agreement was actually there, so it was just the topology property that was wrong. I could have fixed it to changing the attribute from left-to-right to both, using ldapmodify. What I did was to delete the topology segement and put it back. But that ended up with two replication agreements from krb2 to krb4. So I deleted the topology segment, then manually deleted "cn=meTokrb4.cs.rutgers.eduhttp://meTokrb4.cs.rutgers.edu,cn=replica,cn=dc\3Dcs\2Cdc\3Drutgers\2Cdc\3Dedu,cn=mapping tree,cn=config changetype:delete" and added back the topology segment.
It's worth noting that reiniting ldap won't fix issues with replication, because only the user database is synced. Replication agreements are in cn=config. In theory deleting the topology segment and putting it back will fix a lot of issues, but not all of them.
[edithttp://klinzhai.rutgers.edu/mediawiki/index.php?title=Setting_up_backup_server&action=edit§ion=4]Diagnosing permission failures
First, make sure the directory server principals are recognized by the others. E.g. on krb3

kinit -k -t /etc/dirsrv/ds.keytab ldap/[e-mail]

Now do

ldapsearch -Y GSSAPI -h krb1.cs.rutgers.eduhttp://krb1.cs.rutgers.edu uid=hedrick

for both krb1 and 2, to make sure authentiction works. I'd do that on all servers to all servers. If it's working, then the other thing to check is that the principals are in the group "cn=replication managers,cn=sysaccounts,cn=etc,dc=cs,dc=rutgers,dc=edu".

ldapsearch -Y GSSAPI cn="replication managers"

If the data isn't syncing you may want to do that on all servers.

[lejeczek]

On 09/01/18 17:24, Charles Hedrick wrote:
> I also had issues installing a replica under 7.4. Here are 
> my notes. krb4 is the new replica, krb1 and 2 the existing 
> ones.

I'm on Centos, there is something very wrong with freeipa / 
dependencies in 7.4.
I've had four replicas/servers from 7.1 time and just now 
removed one server from domain, all these problems I hit 
earlier were while setting a new domain, but now I see I 
cannot reconnect that one node back to the old, still 
functioning domain, the same errors.
Installing a new servers goes smoothly(?) but adding a 
replica feels like pain in a buttock that does want to go 
away :)

The weirdest thing is that randomness with which client 
installation succeeds, 99% time it fails, and clocks are in 
sync.

[Florence Blanc-Renaud]

On 01/10/2018 12:29 PM, lejeczek via FreeIPA-users wrote:
> On 09/01/18 17:24, Charles Hedrick wrote:
> > I also had issues installing a replica under 7.4. Here are my notes. 
> > krb4 is the new replica, krb1 and 2 the existing ones.
>
> I'm on Centos, there is something very wrong with freeipa / dependencies 
> in 7.4.
> I've had four replicas/servers from 7.1 time and just now removed one 
> server from domain, all these problems I hit earlier were while setting 
> a new domain, but now I see I cannot reconnect that one node back to the 
> old, still functioning domain, the same errors.
> Installing a new servers goes smoothly(?) but adding a replica feels 
> like pain in a buttock that does want to go away :)
>
> The weirdest thing is that randomness with which client installation 
> succeeds, 99% time it fails, and clocks are in sync.
> _______________________________________________
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

Hi,

when the replica installation fails, you need to clean the replication 
agreements before you can re-try replication installation on the same 
machine:
- on the master, ipa-replica-manage del <replica>
(you can do ipa-replica-manage list to make sure that the replica is not 
listed any more).

- check if the host is still in the list of hosts (on the master): ipa 
host-find
You can remove the host with ipa host-del <replica>

When you reach the state where ipa-replica-manage list and ipa host-find 
do not show any more the replica, you can re-install it with 
ipa-client-install and ipa-replica-install.

As you mention issues that do not happen 100% of the time, I would check 
the DNS configuration. Is your IPA client installed with DNS 
autodiscovery or with a fixed list of IPA servers?

Flo

[lejeczek]

On 10/01/18 15:48, Florence Blanc-Renaud wrote:
> On 01/10/2018 12:29 PM, lejeczek via FreeIPA-users wrote:
> > On 09/01/18 17:24, Charles Hedrick wrote:
> > > I also had issues installing a replica under 7.4. Here 
> > > are my notes. krb4 is the new replica, krb1 and 2 the 
> > > existing ones.
> >
> > I'm on Centos, there is something very wrong with freeipa 
> > / dependencies in 7.4.
> > I've had four replicas/servers from 7.1 time and just now 
> > removed one server from domain, all these problems I hit 
> > earlier were while setting a new domain, but now I see I 
> > cannot reconnect that one node back to the old, still 
> > functioning domain, the same errors.
> > Installing a new servers goes smoothly(?) but adding a 
> > replica feels like pain in a buttock that does want to go 
> > away :)
> >
> > The weirdest thing is that randomness with which client 
> > installation succeeds, 99% time it fails, and clocks are 
> > in sync.
> > _______________________________________________
> > FreeIPA-users mailing list -- 
> > [e-mail]
> > To unsubscribe send an email to 
> > [e-mail]
>
> Hi,
>
> when the replica installation fails, you need to clean the 
> replication agreements before you can re-try replication 
> installation on the same machine:
> on the master, ipa-replica-manage del <replica>
>
> (you can do ipa-replica-manage list to make sure that the 
> replica is not listed any more).
> check if the host is still in the list of hosts (on the
>
> master): ipa host-find
> You can remove the host with ipa host-del <replica>
>
> When you reach the state where ipa-replica-manage list and 
> ipa host-find do not show any more the replica, you can 
> re-install it with ipa-client-install and 
> ipa-replica-install.
>
> As you mention issues that do not happen 100% of the time, 
> I would check the DNS configuration. Is your IPA client 
> installed with DNS autodiscovery or with a fixed list of 
> IPA servers?
>
> Flo

When replica installation fails it does not leave anything in:
$ ipa-replica-manage list
There is just one record, one server there. But it gets to 
that point where it leave host entry.
I'd like to think it's very simple, minimalistic setup:
- one newly installed server, it's resolver points to 127.0.0.1
- one client candidate which resolver points directly to 
IPA's dns only.

All these errors, I've just posted a second different error, 
I get in/from this simple setup. It's all 4.5.0 on Centos 7.4

thanks, L.

[lejeczek]

hi,
another, different error this time.

# replica:
..
   [28/40]: adding sasl mappings to the directory
   [29/40]: updating schema
ipa         : CRITICAL Failed to load schema-update.ldif: 
Command '/usr/bin/ldapmodify -v -f 
/usr/share/ipa/schema-update.ldif -H 
ldapi://%2Fvar%2Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK.socket 
-Y EXTERNAL' returned non-zero exit status 50
   [error] CalledProcessError: Command '/usr/bin/ldapmodify 
-v -f /usr/share/ipa/schema-update.ldif -H 
ldapi://%2Fvar%2Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK.socket 
-Y EXTERNAL' returned non-zero exit status 50
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

# replica log:
...
2018-01-10T16:19:20Z DEBUG stderr=ldap_initialize( 
ldapi://%2Fvar%2Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM
-AC-UK.socket/??base )
SASL/EXTERNAL authentication started
SASL username: 
gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
ldap_modify: Insufficient access (50)
         additional info: Insufficient 'write' privilege to 
the 'objectClasses' attribute of entry 'cn=schema
'.

2018-01-10T16:19:20Z CRITICAL Failed to load 
schema-update.ldif: Command '/usr/bin/ldapmodify -v -f /usr/sha
re/ipa/schema-update.ldif -H 
ldapi://%2Fvar%2Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK.socket 
-Y EXTER
NAL' returned non-zero exit status 50
2018-01-10T16:19:20Z DEBUG Traxx.ck (most recent call last):
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 504, in start_creation
     run_step(full_msg, method)
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 494, in run_step
     method()
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
line 490, in __update_schema
     self._ldap_mod("schema-update.ldif")
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 308, in _ldap_mod
     ipautil.run(args, nolog=nologlist)
   File 
"/usr/lib/python2.7/site-packages/ipapython/ipautil.py", 
line 512, in run
     raise CalledProcessError(p.returncode, arg_string, 
str(output))
CalledProcessError: Command '/usr/bin/ldapmodify -v -f 
/usr/share/ipa/schema-update.ldif -H ldapi://%2Fvar%2
Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK.socket -Y 
EXTERNAL' returned non-zero exit status 50

2018-01-10T16:19:20Z DEBUG   [error] CalledProcessError: 
Command '/usr/bin/ldapmodify -v -f /usr/share/ipa/s
chema-update.ldif -H 
ldapi://%2Fvar%2Frun%2Fslapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK.socket 
-Y EXTERNAL' ret
urned non-zero exit status 50

[lejeczek]

On 06/01/18 19:54, lejeczek via FreeIPA-users wrote:
> hi
>
> I'm trying to install replica, process fails:
> ..
>   [3/5]: creating anonymous principal
>   [4/5]: starting the KDC
>   [5/5]: configuring KDC to start on boot
> Done configuring Kerberos KDC (krb5kdc).
> Configuring kadmin
>   [1/2]: starting kadmin
>   [2/2]: configuring kadmin to start on boot
> Done configuring kadmin.
> Configuring directory server (dirsrv)
>   [1/3]: configuring TLS for DS instance
>   [error] RuntimeError: Certificate issuance failed 
> (CA_UNREACHABLE)
> Your system may be partly configured.
> ..
> -- end
>
> and in intall log file:
> ..
> 2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
> PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
> /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
> 2018-01-06T13:50:29Z DEBUG Process finished, return code=0
> 2018-01-06T13:50:29Z DEBUG stdout=
> 2018-01-06T13:50:29Z DEBUG stderr=
> 2018-01-06T13:50:30Z DEBUG certmonger request is in state 
> dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG certmonger request is in state 
> dbus.String(u'CA_UNREACHABLE', variant_level=1)
> 2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 504, in start_creation
>     run_step(full_msg, method)
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
> line 494, in run_step
>     method()
>   File 
> "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
> line 824, in __enable_ssl
>     post_command=cmd)
>   File 
> "/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", 
> line 317, in request_and_wait_for_cert
>     raise RuntimeError("Certificate issuance failed 
> ({})".format(state))
> RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
>
> 2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: 
> Certificate issuance failed (CA_UNREACHABLE)
> 2018-01-06T13:50:35Z DEBUG   File 
> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", 
> line 172, in execute
>     return_value = self.run()
>   File 
> "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
> line 333, in run
>     cfgr.run()
>   File "/usr/lib/python2.7/site-
> ...
> -- end
>
> Would this be that new candidate's problem or some 
> communication issues with existing server? Client 
> installed (kind of)okey though.
> _______________________________________________
> FreeIPA-users mailing list -- 
> [e-mail]
> To unsubscribe send an email to 
> [e-mail]

I might have missed this(if reveals some more?) in dirsrv on 
"working" newly installed server, at the time of - 
ipa-replica-install --no-ntp
...
Configuring directory server (dirsrv)
   [1/3]: configuring TLS for DS instance
   [error] RuntimeError: Certificate issuance failed 
(CA_UNREACHABLE)

Server dirsrv errors log file:
...
[11/Jan/2018:11:42:49.118819569 +0000] - NOTICE - 
NSMMReplicationPlugin - changelog program - _cl5ConstructRUV 
- Rebuilding replication changelog RUV complete.  Result 0 
(Success)
[11/Jan/2018:11:42:49.120916672 +0000] - NOTICE - 
NSMMReplicationPlugin - changelog program - _cl5ConstructRUV 
- Rebuilding the replication changelog RUV, this may take 
several minutes...
[11/Jan/2018:11:42:49.122618751 +0000] - NOTICE - 
NSMMReplicationPlugin - changelog program - _cl5ConstructRUV 
- Rebuilding replication changelog RUV complete.  Result 0 
(Success)
[11/Jan/2018:11:42:49.219688584 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=104 
op=5 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:42:49.242628179 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=105 
op=5 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:42:50.789296435 +0000] - INFO - 
NSMMReplicationPlugin - repl5_tot_run - Beginning total 
update of replica 
"agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)".
[11/Jan/2018:11:42:50.793594364 +0000] - NOTICE - 
NSMMReplicationPlugin - replica_subentry_check - Need to 
create replication keep alive entry <cn=repl keep alive 
4,dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x>
[11/Jan/2018:11:42:50.795313633 +0000] - INFO - 
NSMMReplicationPlugin - replica_subentry_create - add dn: 
cn=repl keep alive 
4,dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x
objectclass: top
objectclass: ldapsubentry
objectclass: extensibleObject
cn: repl keep alive 4
[11/Jan/2018:11:42:53.955962624 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=106 
op=5 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:42:55.159161994 +0000] - INFO - 
NSMMReplicationPlugin - repl5_tot_run - Finished total 
update of replica 
"agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)". 
Sent 471 entries.
[11/Jan/2018:11:42:56.970750501 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=106 
op=6 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:43:02.041747211 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=107 
op=5 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:43:05.054749534 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=107 
op=6 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:43:11.099143389 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=107 
op=7 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:43:23.153766360 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=107 
op=9 replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied
[11/Jan/2018:11:43:47.262418191 +0000] - ERR - 
NSMMReplicationPlugin - 
multimaster_extop_StartNSDS50ReplicationRequest - conn=107 
op=11 
replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": 
Unable to acquire replica: error: permission denied

Does above help to explain as what might be wrong?
many thanks, L.

[Florence Blanc-Renaud]

On 01/11/2018 12:56 PM, lejeczek via FreeIPA-users wrote:
> On 06/01/18 19:54, lejeczek via FreeIPA-users wrote:
> > hi
> >
> > I'm trying to install replica, process fails:
> > ..
> >   [3/5]: creating anonymous principal
> >   [4/5]: starting the KDC
> >   [5/5]: configuring KDC to start on boot
> > Done configuring Kerberos KDC (krb5kdc).
> > Configuring kadmin
> >   [1/2]: starting kadmin
> >   [2/2]: configuring kadmin to start on boot
> > Done configuring kadmin.
> > Configuring directory server (dirsrv)
> >   [1/3]: configuring TLS for DS instance
> >   [error] RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
> > Your system may be partly configured.
> > ..
> > -- end
> >
> > and in intall log file:
> > ..
> > 2018-01-06T13:50:29Z DEBUG args=/usr/bin/certutil -d 
> > /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/ -A -n 
> > PRIVATE.xx.xx.PRIVATE.xx.xx.x IPA CA -t CT,C,C -a -f 
> > /etc/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/pwdfile.txt
> > 2018-01-06T13:50:29Z DEBUG Process finished, return code=0
> > 2018-01-06T13:50:29Z DEBUG stdout=
> > 2018-01-06T13:50:29Z DEBUG stderr=
> > 2018-01-06T13:50:30Z DEBUG certmonger request is in state 
> > dbus.String(u'NEWLY_ADDED_READING_CERT', variant_level=1)
> > 2018-01-06T13:50:35Z DEBUG certmonger request is in state 
> > dbus.String(u'CA_UNREACHABLE', variant_level=1)
> > 2018-01-06T13:50:35Z DEBUG Traxx.ck (most recent call last):
> >   File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 
> > 504, in start_creation
> >     run_step(full_msg, method)
> >   File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 
> > 494, in run_step
> >     method()
> >   File 
> > "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", 
> > line 824, in __enable_ssl
> >     post_command=cmd)
> >   File 
> > "/usr/lib/python2.7/site-packages/ipalib/install/certmonger.py", line 
> > 317, in request_and_wait_for_cert
> >     raise RuntimeError("Certificate issuance failed ({})".format(state))
> > RuntimeError: Certificate issuance failed (CA_UNREACHABLE)
> >
> > 2018-01-06T13:50:35Z DEBUG   [error] RuntimeError: Certificate 
> > issuance failed (CA_UNREACHABLE)
> > 2018-01-06T13:50:35Z DEBUG   File 
> > "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, 
> > in execute
> >     return_value = self.run()
> >   File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", 
> > line 333, in run
> >     cfgr.run()
> >   File "/usr/lib/python2.7/site-
> > ...
> > -- end
> >
> > Would this be that new candidate's problem or some communication 
> > issues with existing server? Client installed (kind of)okey though.
> > _______________________________________________
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to 
> > [e-mail]
>
> I might have missed this(if reveals some more?) in dirsrv on "working" 
> newly installed server, at the time of - ipa-replica-install --no-ntp
> ...
> Configuring directory server (dirsrv)
>    [1/3]: configuring TLS for DS instance
>    [error] RuntimeError: Certificate issuance failed (CA_UNREACHABLE)

I must admit that I'm getting lost among all the errors... Can you 
summarize your topology (for instance server A installed as first IPA 
master, then server B successfully configured as a replica, then server 
C where I tried to run ipa-replica-install but the command failed).

This way we'll be able to sort out the various issues.

Thanks,
Flo
> Server dirsrv errors log file:
> ...
> [11/Jan/2018:11:42:49.118819569 +0000] - NOTICE - NSMMReplicationPlugin
> changelog program - _cl5ConstructRUV - Rebuilding replication
>
> changelog RUV complete.  Result 0 (Success)
> [11/Jan/2018:11:42:49.120916672 +0000] - NOTICE - NSMMReplicationPlugin
> changelog program - _cl5ConstructRUV - Rebuilding the replication
>
> changelog RUV, this may take several minutes...
> [11/Jan/2018:11:42:49.122618751 +0000] - NOTICE - NSMMReplicationPlugin
> changelog program - _cl5ConstructRUV - Rebuilding replication
>
> changelog RUV complete.  Result 0 (Success)
> [11/Jan/2018:11:42:49.219688584 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=104 op=5 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:42:49.242628179 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=105 op=5 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:42:50.789296435 +0000] - INFO - NSMMReplicationPlugin - 
> repl5_tot_run - Beginning total update of replica 
> "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)".
> [11/Jan/2018:11:42:50.793594364 +0000] - NOTICE - NSMMReplicationPlugin
> replica_subentry_check - Need to create replication keep alive entry
>
> <cn=repl keep alive 4,dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x>
> [11/Jan/2018:11:42:50.795313633 +0000] - INFO - NSMMReplicationPlugin - 
> replica_subentry_create - add dn: cn=repl keep alive 
> 4,dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x
> objectclass: top
> objectclass: ldapsubentry
> objectclass: extensibleObject
> cn: repl keep alive 4
> [11/Jan/2018:11:42:53.955962624 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=106 op=5 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:42:55.159161994 +0000] - INFO - NSMMReplicationPlugin - 
> repl5_tot_run - Finished total update of replica 
> "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)". Sent 471 
> entries.
> [11/Jan/2018:11:42:56.970750501 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=106 op=6 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:43:02.041747211 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=107 op=5 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:43:05.054749534 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=107 op=6 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:43:11.099143389 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=107 op=7 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:43:23.153766360 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=107 op=9 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
> [11/Jan/2018:11:43:47.262418191 +0000] - ERR - NSMMReplicationPlugin - 
> multimaster_extop_StartNSDS50ReplicationRequest - conn=107 op=11 
> replica="dc=priv,dc=xx.dc=xx.dc=priv,dc=xx,dc=xx,dc=x": Unable to 
> acquire replica: error: permission denied
>
> Does above help to explain as what might be wrong?
> many thanks, L.
> _______________________________________________
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[lejeczek]

On 11/01/18 17:12, Florence Blanc-Renaud wrote:
> I must admit that I'm getting lost among all the errors... 
> Can you summarize your topology (for instance server A 
> installed as first IPA master, then server B successfully 
> configured as a replica, then server C where I tried to 
> run ipa-replica-install but the command failed).
>
> This way we'll be able to sort out the various issues.
>
> Thanks,
> Flo

I'd like to think it's very simple, minimalistic setup:
- one newly installed server, it's resolver points to 127.0.0.1
- one client candidate which resolver points directly to 
IPA's dns only.
Just one server which installed apparently okey.
Just one replica candidate, client installed okey.

Replica install fails, when it does it leave nothing in 
ipa-replica-manage, only add client installation add host 
record.
...
   [1/3]: configuring TLS for DS instance
   [error] RuntimeError: Certificate issuance failed 
(CA_UNREACHABLE)
Your system may be partly configured.

-- Working Server when replica installation fails
--- The server end, httpd/error_log :
...
[Thu Jan 11 17:20:53.475973 2018] [:error] [pid 2701892] 
ipa: INFO: [jsonserver_kerb] host/dzien.priv.
[e-mail].x: ping(): 
SUCCESS
[Thu Jan 11 17:20:53.527232 2018] [:error] [pid 2701893] 
ipa: INFO: [jsonserver_kerb] host/dzien.priv.
[e-mail].x: 
env((u'version',)): SUCCESS
[Thu Jan 11 17:20:53.573580 2018] [:error] [pid 2701892] 
ipa: INFO: [jsonserver_kerb] host/dzien.priv.
[e-mail].x: 
env((u'fips_mode',)): SUCCESS
[Thu Jan 11 17:21:04.406246 2018] [:error] [pid 2701893] 
ipa: INFO: [jsonserver_kerb] [e-mail].
xx.PRIVATE.xx.xx.x: ping(): SUCCESS
[Thu Jan 11 17:21:04.444042 2018] [:error] [pid 2701892] 
ipa: INFO: [jsonserver_kerb] [e-mail].
xx.PRIVATE.xx.xx.x: ping/1(version=u'2.228'): SUCCESS
[Thu Jan 11 17:21:04.900349 2018] [:error] [pid 2701893] 
ipa: INFO: [jsonserver_kerb] [e-mail].
xx.PRIVATE.xx.xx.x: 
server_conncheck(u'swir.priv.xx.xx.priv.xx.xx.x', 
u'dzien.priv.xx.
xx.priv.xx.xx.x', version=u'2.162'): SUCCESS
[Thu Jan 11 17:21:40.832678 2018] [auth_gssapi:error] [pid 
2702831] [client 10.5.6.17:47072] NO AUTH DATA
  Client did not send any authentication headers, referer: 
https://swir.priv.xx.xx.priv.xx.xx.x
/ipa/xml
[Thu Jan 11 17:21:40.913393 2018] [:error] [pid 2701892] 
ipa: INFO: [xmlserver] 
host/[e-mail].x: 
cert_request(u'MIIEWTCCA0ECAQAwYDErMCkGA1UEChMiUFJJVkFURS5DQ05SLkNFQi5QUklWQVRFLkNBTS5BQy5VSzExMC8GA1UEAxMoZHppZW4ucHJpdmF0ZS5jY25yLmNlYi5wcml2YXRlLmNhbS5hYy51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK81UM4J4041W+ujikOQrplTcUjjmMHOWkE8YbVwjZvH2AYhP7tn+jZ7GYWZuv//j4tfFEEiIyV94hL85xjhXCwkLdR1R6CaswD1SpwOkQGOOICol1aV2tX1+o9Q3Q4gDLMBvxX3+s+l39l7dgDqj6ZaueH8E7H+W9Oj+w3+E8mR5Zy/kG6X6WNtwbqyf6fZwshRO26O4aUNlxMDpeLN5mDoRWBeeHiWoFBXwYXqysHoVLFN4urpH7wHKSEkdW7Vaj/1HXGutOV5HsaBqeGzdCiwlrBFDE9maU7HhyBxxGEit11ejGxoP2hjxyO0l0tfBuqJjBkGZDvgB3501jGgHQ8CAwEAAaCCAbIwJQYJKoZIhvcNAQkUMRgeFgBTAGUAcgB2AGUAcgAtAEMAZQByAHQwggGHBgkqhkiG9w0BCQ4xggF4MIIBdDCCAQwGA1UdEQEBAASCAQAwgf2CKGR6aWVuLnByaXZhdGUuY2Nuci5jZWIucHJpdmF0ZS5jYW0uYWMudWugYAYKKwYBBAGCNxQCA6BSDFBsZGFwL2R6aWVuLnByaXZhdGUuY2Nuci5jZWIucHJpdmF0ZS5jYW0uYWMudWtAUFJJVkFURS5DQ05SLkNFQi5QUklWQVRFLkNBTS5BQy5VS6BvBgYrBgEFAgKgZTBjoCQbIlBSSVZBVEUuQ0NOUi5DRUIuUFJJVkFURS5DQU0uQUMuVUuhOzA5oAMCAQGhMjAwGwRsZGFwGyhkemllbi5wcml2YXRlLmNjbnIuY2ViLnByaXZhdGUuY2FtLmFjLnVrMAwGA1UdEwEB/wQCMAAwIAYDVR0OAQEABBYEFA1WfK69+iWx/R7b8kWpd/4M3QtrMDIGCSsGAQQBgjcUAgEBAAQiHiAAYwBhAEkAUABBAHMAZQByAHYAaQBjAGUAQwBlAHIAdDANBgkqhkiG9w0BAQsFAAOCAQEAXBhrwG29M+UEWZsWjIJWRHEQ6bAdpTWJVP7P9eXu11Krd/VrCb/qgz7/68GWkKCDpINLB8P7SGWoUwI4zjn1YJNn9jyVELryLn4MHfQ/3W3mVZcK6FPtRqmpwZ6q0TEG/ZpBftWlpI8AkhruGrx8bJ/pGo3UJgY/5QajMjsuuysvoukVxeYmQJDCH36Hdfw6+hYg3XQfmJ2WgLjnVaA0v3e9gKYf0gObwSbKF1ZqF6pVw9D9h8BgTITB8PO6aQvYCeWY6H9w/Dc3V77pPubTrScN6JJFZEyTWPK2GijpO7E+4nV5bnHNlJ4LpAZRYH0lzcVok/YNsRv8bupoFoChNQ==', 
profile_id=u'caIPAserviceCert', 
principal=u'ldap/[e-mail].x', 
add=True, version=u'2.51'): NetworkError

--- The server, tomcat, if I do:
$ egrep '(warn|error|fail|canno)' 
/var/log/pki/pki-tomcat/ca/debug
I see many:
[11/Jan/2018:17:12:55][localhost-startStop-1]: init: before 
makeConnection errorIfDown is false
[11/Jan/2018:17:12:55][localhost-startStop-1]: 
makeConnection: errorIfDown false
[11/Jan/2018:17:12:55][localhost-startStop-1]: init: before 
makeConnection errorIfDown is false
[11/Jan/2018:17:12:55][localhost-startStop-1]: 
makeConnection: errorIfDown false

But time stamps do not see to correspond to what's in 
httpd/error_log
Also cannot see something like "PKIRealm: Authenticating 
certificate chain" around the time of replica installation.

Should I also be looking at  /var/log/dirsrv/xx/erros mabye?

[lejeczek]

On 11/01/18 17:12, Florence Blanc-Renaud wrote:
> I must admit that I'm getting lost among all the errors... 
> Can you summarize your topology (for instance server A 
> installed as first IPA master, then server B successfully 
> configured as a replica, then server C where I tried to 
> run ipa-replica-install but the command failed).
>
> This way we'll be able to sort out the various issues.
>
> Thanks,
> Flo

Ok, dirsrv errors just in case,
all the server logged during replica failed installation:

$ tailf 
/var/log/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/errors
[11/Jan/2018:18:01:51.302445627 +0000] - ERR - 
NSMMReplicationPlugin - bind_and_check_pwp - 
agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389) - 
Replication bind with GSSAPI auth failed: LDAP error 49 
(Invalid credentials) ()
[11/Jan/2018:18:01:51.366234558 +0000] - INFO - 
NSMMReplicationPlugin - bind_and_check_pwp - 
agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389): 
Replication bind with GSSAPI auth resumed
[11/Jan/2018:18:01:52.914160480 +0000] - INFO - 
NSMMReplicationPlugin - repl5_tot_run - Beginning total 
update of replica 
"agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)".
[11/Jan/2018:18:01:57.349282726 +0000] - INFO - 
NSMMReplicationPlugin - repl5_tot_run - Finished total 
update of replica 
"agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)". 
Sent 554 entries.
[11/Jan/2018:18:02:02.381314331 +0000] - WARN - 
NSMMReplicationPlugin - acquire_replica - 
agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389): 
Unable to receive the response for a startReplication 
extended operation to consumer (Can't contact LDAP server). 
Will retry later.
[11/Jan/2018:18:02:05.449923136 +0000] - INFO - 
NSMMReplicationPlugin - bind_and_check_pwp - 
agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389): 
Replication bind with GSSAPI auth resumed

[Rob Crittenden]

lejeczek via FreeIPA-users wrote:
> On 11/01/18 17:12, Florence Blanc-Renaud wrote:
> > I must admit that I'm getting lost among all the errors... Can you
> > summarize your topology (for instance server A installed as first IPA
> > master, then server B successfully configured as a replica, then
> > server C where I tried to run ipa-replica-install but the command
> > failed).
> >
> > This way we'll be able to sort out the various issues.
> >
> > Thanks,
> > Flo
>
> Ok, dirsrv errors just in case,
> all the server logged during replica failed installation:
>
> $ tailf /var/log/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/errors
> [11/Jan/2018:18:01:51.302445627 +0000] - ERR - NSMMReplicationPlugin -
> bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> (dzien:389) - Replication bind with GSSAPI auth failed: LDAP error 49
> (Invalid credentials) ()
> [11/Jan/2018:18:01:51.366234558 +0000] - INFO - NSMMReplicationPlugin -
> bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> (dzien:389): Replication bind with GSSAPI auth resumed
> [11/Jan/2018:18:01:52.914160480 +0000] - INFO - NSMMReplicationPlugin -
> repl5_tot_run - Beginning total update of replica
> "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)".
> [11/Jan/2018:18:01:57.349282726 +0000] - INFO - NSMMReplicationPlugin -
> repl5_tot_run - Finished total update of replica
> "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)". Sent 554
> entries.
> [11/Jan/2018:18:02:02.381314331 +0000] - WARN - NSMMReplicationPlugin -
> acquire_replica - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> (dzien:389): Unable to receive the response for a startReplication
> extended operation to consumer (Can't contact LDAP server). Will retry
> later.
> [11/Jan/2018:18:02:05.449923136 +0000] - INFO - NSMMReplicationPlugin -
> bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> (dzien:389): Replication bind with GSSAPI auth resumed

Are you absolutely sure the network ports are open in both directions?

You aren't using the --skip-conncheck argument are you?

rob

[lejeczek]

On 11/01/18 20:28, Rob Crittenden wrote:
> lejeczek via FreeIPA-users wrote:
> > On 11/01/18 17:12, Florence Blanc-Renaud wrote:
> > > I must admit that I'm getting lost among all the errors... Can you
> > > summarize your topology (for instance server A installed as first IPA
> > > master, then server B successfully configured as a replica, then
> > > server C where I tried to run ipa-replica-install but the command
> > > failed).
> > >
> > > This way we'll be able to sort out the various issues.
> > >
> > > Thanks,
> > > Flo
> >
> > Ok, dirsrv errors just in case,
> > all the server logged during replica failed installation:
> >
> > $ tailf /var/log/dirsrv/slapd-PRIVATE-xx.xx.PRIVATE-CAM-AC-UK/errors
> > [11/Jan/2018:18:01:51.302445627 +0000] - ERR - NSMMReplicationPlugin -
> > bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> > (dzien:389) - Replication bind with GSSAPI auth failed: LDAP error 49
> > (Invalid credentials) ()
> > [11/Jan/2018:18:01:51.366234558 +0000] - INFO - NSMMReplicationPlugin -
> > bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> > (dzien:389): Replication bind with GSSAPI auth resumed
> > [11/Jan/2018:18:01:52.914160480 +0000] - INFO - NSMMReplicationPlugin -
> > repl5_tot_run - Beginning total update of replica
> > "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)".
> > [11/Jan/2018:18:01:57.349282726 +0000] - INFO - NSMMReplicationPlugin -
> > repl5_tot_run - Finished total update of replica
> > "agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x" (dzien:389)". Sent 554
> > entries.
> > [11/Jan/2018:18:02:02.381314331 +0000] - WARN - NSMMReplicationPlugin -
> > acquire_replica - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> > (dzien:389): Unable to receive the response for a startReplication
> > extended operation to consumer (Can't contact LDAP server). Will retry
> > later.
> > [11/Jan/2018:18:02:05.449923136 +0000] - INFO - NSMMReplicationPlugin -
> > bind_and_check_pwp - agmt="cn=meTodzien.priv.xx.xx.priv.xx.xx.x"
> > (dzien:389): Replication bind with GSSAPI auth resumed
>
> Are you absolutely sure the network ports are open in both directions?
>
> You aren't using the --skip-conncheck argument are you?
>
> rob

I'm double posting.. beware
Jesus freaking Christ.. (this comes after I produced a whole 
litany of of bad words in my own language), sorry.
It almost drove me insane! no, really!

all these problems, all these errors, all because of my 
root's umask 027
Now having replica installed, I'll see how two servers 
behave in my simple domain.

Guys, make it a very first check in installer code and make 
that installer fail, and.. push out a new release with that 
little fix like... yesterday(do not wait till it's properly 
fixed) You can still save lives! :)