Replication error

[Bret Wortman]

I've also just realized that replication appears to have ceased; I have 
entries in some IPA servers but not all.

[root@zsipa ~]# ipa-replica-manage list
Directory Manager password:

zsipa.damascusgrp.com: master
zsipa2.damascusgrp.com: master
zsipa3.damascusgrp.com: master
[root@zsipa ~]# ipa-replica-manage list zsipa.damascusgrp.com
Directory Manager password:

zsipa3.damascusgrp.com: replica
   last init status: None
   last init ended: 1970-01-01 00:00:00+00:00
   last update status: Error (19) Replication error acquiring replica: 
Replica has different database generation ID, remote replica may need to 
be initialized (RUV error)
   last update ended: 1970-01-01 00:00:00+00:00
[root@zsipa ~]#

Only zsipa3 is listed as a replica anywhere, and it's not a functioning 
one. I can set up replication between zsipa and zsipa2, but is there a 
good way to bring zsipa3 back in line as well?

The background is that we attempted to do a rolling update of our IPA 
servers by bringing in a new server, zsipa2, and then upgrading each of 
the other two from Fedora to Centos 7 and then initialized them as 
replicas of zsipa2. But apparently, this didn't work as we had thought. 
So add replication errors to the certificate issue I'm still trying to 
run to ground.

[Striker Leggette]

You can try to force a re-init from the broken server:

# kinit admin
# ipa-replica-manage re-initialize --from workinghost1.example.com

On 06/05/2017 11:07 AM, Bret Wortman via FreeIPA-users wrote:
> I've also just realized that replication appears to have ceased; I 
> have entries in some IPA servers but not all.
>
> [root@zsipa ~]# ipa-replica-manage list
> Directory Manager password:
>
> zsipa.damascusgrp.com: master
> zsipa2.damascusgrp.com: master
> zsipa3.damascusgrp.com: master
> [root@zsipa ~]# ipa-replica-manage list zsipa.damascusgrp.com
> Directory Manager password:
>
> zsipa3.damascusgrp.com: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (19) Replication error acquiring replica: 
> Replica has different database generation ID, remote replica may need 
> to be initialized (RUV error)
>   last update ended: 1970-01-01 00:00:00+00:00
> [root@zsipa ~]#
>
> Only zsipa3 is listed as a replica anywhere, and it's not a 
> functioning one. I can set up replication between zsipa and zsipa2, 
> but is there a good way to bring zsipa3 back in line as well?
>
> The background is that we attempted to do a rolling update of our IPA 
> servers by bringing in a new server, zsipa2, and then upgrading each 
> of the other two from Fedora to Centos 7 and then initialized them as 
> replicas of zsipa2. But apparently, this didn't work as we had 
> thought. So add replication errors to the certificate issue I'm still 
> trying to run to ground.
>
> -- 
> *Bret Wortman*
> Damascus Products
> ph/fax: [phone]
> Wrap Buddies <wrapbuddies.co/store> now available for preorder!
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Bret Wortman]

I think I (finally) figured out what went wrong, but I have no idea how 
to proceed. Somehow I missed an error during the setup, and so now I 
think whatever CA I had has been clobbered and rendered useless (since 
it was wiped and reinstalled after this error occurred):

Please don't read too much into any typos below -- I had to type this by 
hand as the originals are on an internal development network and this 
was actually faster than transferring files in this direction.

2017-03-29T12:01:22Z DEBUG cert valid True for 
"CN=zsipa.damascusgrp.com,O=damascusgrp.com"
2017-03-29T12:01:22Z DEBUG handshake complete, peer = 192.168.208.53:8443
2017-03-29T12:01:22Z DEBUG Protocol: TLS1.2
2017-03-29T12:01:22Z DEBUG Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
2017-03-29T12:01:22Z DEBUG response status 204
2017-03-29T12:01:22Z DEBUG response headers {'set-cookie': 
'JSESSIONID=7B6440F45777C030B70AC2CCEE7CE780; Path=/ca/; Secure; 
HttpOnlhy', 'expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'server': 
'Apache-Coyote/1.1', 'cache-control': 'private', 'date': 'Wed, 29 Mar 
2017 12:01:21 GMT', 'content-type': 'application/xml'}
2017-03-29T12:01:22Z DEBUG response body ''
2017-03-29T12:01:22Z DEBUG Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 449, in start_creation
     run_step(full_msg, method)
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 439, in run_step
     method()
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
1887, in import_included_profiles
     _create_dogtag_profile(profile_id, profile_data, overwrite=True)
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
2008, in _create_dogtag_profile
     profile_api.update_profile(profile_id, profile_data)
   File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", 
line 2147, in update_profile
     body=profile_data
   File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", 
line 2106, in _ssldo
     % {'status': status, 'explanation': explanation}
RemoteRetrieveError: Non-2xx reponse from CA REST API: 404.

2017-03-29T12:01:22Z DEBUG   [error] RemoteRetrieveError: Non-2xx 
response from CA REST API: 404.
2017-03-29T12:01:22Z DEBUG   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", 
line 752, in run_script
     return_value = main_function()

File "/usr/sbin/ipa-ca-install", line 302, in main
     install(safe_options, options, filename)

File "/usr/sbin/ipa-ca-install", line 242, in install
     install_replica(safe_options, options, filename)

File "/usr/sbin/ipa-ca-install", line 204, in install_replica
     ca.install(True, config, options)

File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 
118, in install
     install_step_0(standalone, replica_config, options)

File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 
140, in install_step_0
     ra_p12=getattr(options, 'ra_p12', None)

File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
1562, in install_replica_ca
     subject_base=config.subject_base)

File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
437, in configure_instance
     self.start_creation(runtime=210)

File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 449, in start_creation
     run_step(full_msg, method)

File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", 
line 439, in run_step
     method()

File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
1887, in import_included_profiles
     _create_dogtag_profile(profile_id, profile_data, overwrite=True)

File 
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
2008, in _create_dogtag_profile
     profile_api.update_profile(profile_id, profile_data)

File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", 
line 2147, in update_profile
     body=profile_data

File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", 
line 2106, in _ssldo
     % {'status': status, 'explanation': explanation}

2017-03-29T12:01:22Z DEBUG The iopa-ca-install command failed, 
exception: RemoteRetrieveError: Non-2xx response from CA REST API: 404.

On 06/05/2017 11:07 AM, Bret Wortman wrote:
> I've also just realized that replication appears to have ceased; I 
> have entries in some IPA servers but not all.
>
> [root@zsipa ~]# ipa-replica-manage list
> Directory Manager password:
>
> zsipa.damascusgrp.com: master
> zsipa2.damascusgrp.com: master
> zsipa3.damascusgrp.com: master
> [root@zsipa ~]# ipa-replica-manage list zsipa.damascusgrp.com
> Directory Manager password:
>
> zsipa3.damascusgrp.com: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (19) Replication error acquiring replica: 
> Replica has different database generation ID, remote replica may need 
> to be initialized (RUV error)
>   last update ended: 1970-01-01 00:00:00+00:00
> [root@zsipa ~]#
>
> Only zsipa3 is listed as a replica anywhere, and it's not a 
> functioning one. I can set up replication between zsipa and zsipa2, 
> but is there a good way to bring zsipa3 back in line as well?
>
> The background is that we attempted to do a rolling update of our IPA 
> servers by bringing in a new server, zsipa2, and then upgrading each 
> of the other two from Fedora to Centos 7 and then initialized them as 
> replicas of zsipa2. But apparently, this didn't work as we had 
> thought. So add replication errors to the certificate issue I'm still 
> trying to run to ground.
>
> -- 
> *Bret Wortman*
> Damascus Products
> ph/fax: [phone]
> Wrap Buddies <wrapbuddies.co/store> now available for preorder!