Hi, 

I just wanted thank Rob for the help and give a conclusion to this thread, in case anyone else needs to replicate from some old version like 4.2

I first wanted to replicate from 4.2.4 to 4.8.4 (provided by CentOS 8). Just not possible (4.8 doesn't support replication from level 0).
So I decided to first replicate from 4.2 to 4.6.6 (provided by CentOS 7). I got almost at the end, but I found too many incompatibilities. Not worth the effort.
My next attempt was from 4.2.4 to 4.4.4 (provided by FC25). This succeeded, but had to upgrade nss to get past https://pagure.io/freeipa/issue/4676

Lesson learned: don't try to skip too many releases.

On Fri, 24 Jul 2020 at 15:33, Roberto Cornacchia <roberto.cornacchia@gmail.com> wrote:
I actually went further - I'm really close now.

Hinted by https://bugzilla.redhat.com/show_bug.cgi?id=1158410, I adjusted 
/usr/share/pki/server/conf/ciphers.info 
/usr/lib/python2.7/site-packages/pki/server/deployment/pkiparser.py 
to use the same ciphers as ipa01.

This worked, and it got almost to the end (with one step in between - I needed to open port 8443 on ipa01):

[root@ipa02 ~]# ipa-ca-install replica-info-ipa02.hq.spinque.com.gpg
Password for admin@HQ.SPINQUE.COM:
Directory Manager (existing master) password:

Run connection check to master
Connection check OK
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/27]: configuring certificate server instance
  [2/27]: reindex attributes
  [3/27]: exporting Dogtag certificate store pin
  [4/27]: stopping certificate server instance to update CS.cfg
  [5/27]: backing up CS.cfg
  [6/27]: disabling nonces
  [7/27]: set up CRL publishing
  [8/27]: enable PKIX certificate path discovery and validation
  [9/27]: starting certificate server instance
  [10/27]: setting audit signing renewal to 2 years
  [11/27]: restarting certificate server
  [12/27]: authorizing RA to modify profiles
  [13/27]: authorizing RA to manage lightweight CAs
  [14/27]: Ensure lightweight CAs container exists
  [15/27]: Ensuring backward compatibility
  [16/27]: configure certificate renewals
  [17/27]: configure Server-Cert certificate renewal
  [18/27]: Configure HTTP to proxy connections
  [19/27]: restarting certificate server
  [20/27]: updating IPA configuration
  [21/27]: enabling CA instance
  [22/27]: exposing CA instance on LDAP
  [23/27]: migrating certificate profiles to LDAP
  [24/27]: importing IPA certificate profiles
  [25/27]: adding default CA ACL
  [26/27]: adding 'ipa' CA entry
  [error] HTTPRequestError: Request failed with status 404: Non-2xx response from CA REST API: 404.

The failing call was https://ipa01.hq.spinque.com:8443/ca/rest/authorities/host-authority.
This comes after a successful POST https://ipa01.hq.spinque.com:8443/ca/rest/profiles/KDCs_PKINIT_Certs?action=enable

The subpath /authorities doesn't seem to exist on ipa01 - probably it changed.

Is there a workaround here?

If not: it was almost at the end. Is it missing crucial steps or can I get away with it?

The CA is up, I'm just not sure if it is safe to use.
Can I check its health further than the following?

[root@ipa02 ~]# ipa server-role-find --role "CA server"
----------------------
2 server roles matched
----------------------
  Server name: ipa02.hq.spinque.com
  Role name: CA server
  Role status: enabled

  Server name: ipa01.hq.spinque.com
  Role name: CA server
  Role status: enabled
----------------------------
Number of entries returned 2
----------------------------



On Fri, 24 Jul 2020 at 13:20, Roberto Cornacchia <roberto.cornacchia@gmail.com> wrote:
Thanks Rob.

Skipping the key checks got me past that error. So the connection test passes!

Unfortunately now I have a cipher issue.

[root@ipa02 ~]# ipa-ca-install replica-info-ipa02.hq.spinque.com.gpg
Directory Manager (existing master) password:

Run connection check to master
Connection check OK
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/27]: configuring certificate server instance
ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpkUNbPC' returned non-zero exit status 1
ipaserver.install.dogtaginstance: CRITICAL See the installation logs and the following files/directories for more information:
ipaserver.install.dogtaginstance: CRITICAL   /var/log/pki/pki-tomcat

/var/log/pki/pki-tomcat/ca/debug

[http-bio-8443-exec-3]: ConfigurationUtils: GET https://ipa01.hq.spinque.com:443/ca/admin/ca/getCertChain
javax.ws.rs.ProcessingException: Unable to invoke request
        at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:287)
...
Caused by: java.io.IOException: SocketException cannot write on socket
        at org.mozilla.jss.ssl.SSLSocket.write(SSLSocket.java:1503)

ipa01:/var/log/httpd/error_log says:
[:error] [pid 18129] SSL Library Error: -12286 No common encryption algorithm(s) with client

I guess it makes sense, old ciphers have been disabled in the newer release.

Testing with openssl from ipa02 against ipa01, I found only these being accepted:
AES128-SHA
DES-CBC3-SHA
RC4-SHA
RC4-MD5

How can I temporarily make ipa-ca-install accept old ciphers?
Before running ipa-ca-install there is even no pki-tomcat configured on ipa02, but running it fails. 

Any idea?

On Fri, 24 Jul 2020 at 00:46, Rob Crittenden <rcritten@redhat.com> wrote:
Roberto Cornacchia via FreeIPA-users wrote:
> Hi Rob,
>
> Thanks for the tip.
>
> I don't see errors that I've found before, but quite some errors.
>
> In attachment is the result of 
> grep -v SUCCESS /var/log/httpd/error_log
> for today.

IPA stopped using memcached in I think version 4.5.0. I guess the key
size in the session grew since then.

I'm not sure what the best workaround is. On the 4.2 servers you could
try to modify /usr/lib/python*/site-packages/ipaserver/session.py and find:

self.mc = memcache.Client(self.servers, debug=0)

Add check_keys=False to that initialization to not check sizing. That
could have other unintended consequences that I'm not aware of.

Restart httpd after making this change.

> I've also tried to replicate the error that I got with
> ipa-replica-install, during the server upgrade step.
> I ran ipa-server-upgrade -v on ipa02, and got the same error
> "ipaserver.install.ldapupdate: ERROR    Add failure attribute "cn" not
> allowed".

/var/log/ipaserver-upgrade.log should have more context.

>
> I also see something else that is strane in the output
> of ipa-server-upgrade -v:
>
> Failed to check CA status: cannot connect to
> 'http://ipa01.hq.spinque.com:8080/ca/admin/ca/getStatus': [Errno 113] No
> route to host
>
> I wonder why 8080. Shouldn't this be on 80?

Try opening port 8080. It tries to contact the CA directly and not
through the Apache proxy.

>
> [root@ipa02 ~]# curl
> 'http://ipa01.hq.spinque.com:8080/ca/admin/ca/getStatus'
> curl: (7) Failed connect to ipa01.hq.spinque.com:8080
> <http://ipa01.hq.spinque.com:8080>; No route to host
>
> [root@ipa02 ~]# curl 'http://ipa01.hq.spinque.com/ca/admin/ca/getStatus'
> <?xml version="1.0" encoding="UTF-8"
> standalone="no"?><XMLResponse><State>1</State><Type>CA</Type><Status>running</Status><Version>10.2.6-20.fc23</Version></XMLResponse>
>
> Roberto
>
> On Thu, 23 Jul 2020 at 19:08, Rob Crittenden <rcritten@redhat.com
> <mailto:rcritten@redhat.com>> wrote:
>
>     Roberto Cornacchia via FreeIPA-users wrote:
>     > ipa-replica-conncheck fails with --auto-master-check (used by
>     > ipa-ca-install), but not without:
>     >
>     >
>     > [root@ipa02 ~]# /usr/sbin/ipa-replica-conncheck --master
>     > ipa01.hq.spinque.com <http://ipa01.hq.spinque.com>
>     <http://ipa01.hq.spinque.com> --auto-master-check
>     > --realm HQ.SPINQUE.COM <http://HQ.SPINQUE.COM>
>     <http://HQ.SPINQUE.COM> --hostname
>     > ipa02.hq.spinque.com <http://ipa02.hq.spinque.com>
>     <http://ipa02.hq.spinque.com>
>     > Check connection from replica to remote master
>     'ipa01.hq.spinque.com <http://ipa01.hq.spinque.com>
>     > <http://ipa01.hq.spinque.com>':
>     >    Directory Service: Unsecure port (389): OK
>     >    Directory Service: Secure port (636): OK
>     >    Kerberos KDC: TCP (88): OK
>     >    Kerberos Kpasswd: TCP (464): OK
>     >    HTTP Server: Unsecure port (80): OK
>     >    HTTP Server: Secure port (443): OK
>     >
>     > The following list of ports use UDP protocoland would need to be
>     > checked manually:
>     >    Kerberos KDC: UDP (88): SKIPPED
>     >    Kerberos Kpasswd: UDP (464): SKIPPED
>     >
>     > Connection from replica to master is OK.
>     > Start listening on required ports for remote master check
>     > 389 tcp: Failed to bind
>     > 636 tcp: Failed to bind
>     > 88 tcp: Failed to bind
>     > 88 udp: Failed to bind
>     > 464 tcp: Failed to bind
>     > 464 udp: Failed to bind
>     > 80 tcp: Failed to bind
>     > 443 tcp: Failed to bind
>     > Get credentials to log in to remote master
>     > Check RPC connection to remote master
>     > trying https://ipa01.hq.spinque.com/ipa/session/json
>     > *Connection to https://ipa01.hq.spinque.com/ipa/session/json
>     failed with
>     > <ProtocolError for ipa01.hq.spinque.com/ipa/session/json
>     <http://ipa01.hq.spinque.com/ipa/session/json>
>     > <http://ipa01.hq.spinque.com/ipa/session/json>: 500 Internal
>     Server Error>*
>     > trying https://ipa02.hq.spinque.com/ipa/session/json
>     > [try 1]: Forwarding 'schema' to json server
>     > 'https://ipa02.hq.spinque.com/ipa/session/json'
>     > trying https://ipa01.hq.spinque.com/ipa/session/json
>     > Connection to https://ipa01.hq.spinque.com/ipa/session/json failed
>     with
>     > <ProtocolError for ipa01.hq.spinque.com/ipa/session/json
>     <http://ipa01.hq.spinque.com/ipa/session/json>
>     > <http://ipa01.hq.spinque.com/ipa/session/json>: 500 Internal
>     Server Error>
>     > trying https://ipa02.hq.spinque.com/ipa/session/json
>     > [try 1]: Forwarding 'ping/1' to json server
>     > 'https://ipa02.hq.spinque.com/ipa/session/json'
>     > Execute check on remote master
>     > [try 1]: Forwarding 'server_conncheck' to json server
>     > 'https://ipa02.hq.spinque.com/ipa/session/json'
>     > *ERROR: Remote master check failed with following error message(s):
>     > invalid 'cn': must be "ipa02.hq.spinque.com
>     <http://ipa02.hq.spinque.com> <http://ipa02.hq.spinque.com>"*
>     >
>     >
>     > Now, without --auto-master-check:
>     >
>     > On ipa02 (I suppose the many "Failed to bind" below are expected?):
>     > [root@ipa02 ~]# /usr/sbin/ipa-replica-conncheck --master
>     > ipa01.hq.spinque.com <http://ipa01.hq.spinque.com>
>     <http://ipa01.hq.spinque.com>  --realm
>     > HQ.SPINQUE.COM <http://HQ.SPINQUE.COM> <http://HQ.SPINQUE.COM>
>     --hostname ipa02.hq.spinque.com <http://ipa02.hq.spinque.com>
>     > <http://ipa02.hq.spinque.com>
>     > Check connection from replica to remote master
>     'ipa01.hq.spinque.com <http://ipa01.hq.spinque.com>
>     > <http://ipa01.hq.spinque.com>':
>     >    Directory Service: Unsecure port (389): OK
>     >    Directory Service: Secure port (636): OK
>     >    Kerberos KDC: TCP (88): OK
>     >    Kerberos Kpasswd: TCP (464): OK
>     >    HTTP Server: Unsecure port (80): OK
>     >    HTTP Server: Secure port (443): OK
>     >
>     > The following list of ports use UDP protocoland would need to be
>     > checked manually:
>     >    Kerberos KDC: UDP (88): SKIPPED
>     >    Kerberos Kpasswd: UDP (464): SKIPPED
>     >
>     > Connection from replica to master is OK.
>     > Start listening on required ports for remote master check
>     > 389 tcp: Failed to bind
>     > 636 tcp: Failed to bind
>     > 88 tcp: Failed to bind
>     > 88 udp: Failed to bind
>     > 464 tcp: Failed to bind
>     > 464 udp: Failed to bind
>     > 80 tcp: Failed to bind
>     > 443 tcp: Failed to bind
>     > Listeners are started. Use CTRL+C to terminate the listening part
>     after
>     > the test.
>     >
>     > Please run the following command on remote master:
>     > /usr/sbin/ipa-replica-conncheck --replica ipa02.hq.spinque.com
>     <http://ipa02.hq.spinque.com>
>     > <http://ipa02.hq.spinque.com>
>     > ^C
>     > Cleaning up...
>     >
>     > On ipa01:
>     > [root@ipa01 ~]# /usr/sbin/ipa-replica-conncheck --replica
>     > ipa02.hq.spinque.com <http://ipa02.hq.spinque.com>
>     <http://ipa02.hq.spinque.com>
>     > Check connection from master to remote replica
>     'ipa02.hq.spinque.com <http://ipa02.hq.spinque.com>
>     > <http://ipa02.hq.spinque.com>':
>     >    Directory Service: Unsecure port (389): OK
>     >    Directory Service: Secure port (636): OK
>     >    Kerberos KDC: TCP (88): OK
>     >    Kerberos KDC: UDP (88): WARNING
>     >    Kerberos Kpasswd: TCP (464): OK
>     >    Kerberos Kpasswd: UDP (464): WARNING
>     >    HTTP Server: Unsecure port (80): OK
>     >    HTTP Server: Secure port (443): OK
>     > The following UDP ports could not be verified as open: 88, 464
>     > This can happen if they are already bound to an application
>     > and ipa-replica-conncheck cannot attach own UDP responder.
>     >
>     > Connection from master to replica is OK.
>     >
>     >
>     >
>     > On Thu, 23 Jul 2020 at 15:15, Roberto Cornacchia
>     > <roberto.cornacchia@gmail.com
>     <mailto:roberto.cornacchia@gmail.com>
>     <mailto:roberto.cornacchia@gmail.com
>     <mailto:roberto.cornacchia@gmail.com>>> wrote:
>     >
>     >     Hi, 
>     >
>     >     I have successfully created a replica from a 4.2.4 master (ipa01)
>     >     into a new 4.6.6 master (ipa02).
>     >
>     >     I did it without --setup-ca option (because it had failed), so the
>     >     only CA is still on the 4.2.4 server (ipa01).
>     >
>     >     When I try to setup theCA on ipa02 (the same replica file was used
>     >     with ipa-replica-install), this fails:
>     >
>     >     $ ipa-ca-install replica-info-ipa02.hq.spinque.com.gpg 
>     >     Directory Manager (existing master) password:
>     >
>     >     Run connection check to master
>     >
>     >     Your system may be partly configured.
>     >     Run /usr/sbin/ipa-server-install --uninstall to clean up.
>     >
>     >     Connection check failed!
>     >     See /var/log/ipareplica-conncheck.log for more information.
>     >     If the check results are not valid it can be skipped with
>     >     --skip-conncheck parameter.
>     >
>     >     The log of conncheck (generated by ipa-ca-install) is in
>     attachment.
>     >     In there, I can see a couple of things going wrong:
>     >
>     >     ProtocolError: <ProtocolError for
>     >     ipa01.hq.spinque.com/ipa/session/json
>     <http://ipa01.hq.spinque.com/ipa/session/json>
>     >     <http://ipa01.hq.spinque.com/ipa/session/json>: 500 Internal
>     Server
>     >     Error>
>     >     ...
>     >     2020-07-23T12:20:50Z ERROR ERROR: Remote master check failed with
>     >     following error message(s):
>     >     invalid 'cn': must be "ipa02.hq.spinque.com
>     <http://ipa02.hq.spinque.com>
>     >     <http://ipa02.hq.spinque.com>"
>     >
>     >     Not sure if relevant, but also ipa-replica-install, though it
>     >     completed successfully, gave this error:
>     >
>     >     Upgrading IPA:. Estimated time: 1 minute 30 seconds
>     >       [1/10]: stopping directory server
>     >       [2/10]: saving configuration
>     >       [3/10]: disabling listeners
>     >       [4/10]: enabling DS global lock
>     >       [5/10]: disabling Schema Compat
>     >       [6/10]: starting directory server
>     >       [7/10]: upgrading server
>     >     ipaserver.install.ldapupdate: ERROR    Add failure attribute "cn"
>     >     not allowed
>     >       [8/10]: stopping directory server
>     >       [9/10]: restoring configuration
>     >       [10/10]: starting directory server
>     >
>     >
>     >     Could you please help me find the issue?
>
>     Look on ipa01.hq.spinque.com <http://ipa01.hq.spinque.com> in
>     /var/log/httpd/error_log for those
>     internal errors.
>
>     rob
>
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
>