Re: Cert Issue

[Rob Crittenden]

Randy Morgan wrote:
> On 9/9/2019 11:31 AM, Rob Crittenden wrote:
> > Randy Morgan via FreeIPA-users wrote:
> > > We have been working to solve an expired certificate issue in IPA.
> > > There is an open ticket in Red Hat supportCASE 02438518.  We have tried
> > > many things but so far have had no luck getting the certs to update.
> > > Currently the system is running RHEL 8.0 and IPA 4.7.1.
> > >
> > > pki-server cert-fix -n 'subsystemCert cert-pki-ca' -d
> > > /var/lib/pki/pki-tomcat/alias/ -C /root/passwd -vvv
> > > INFO: Loading instance: pki-tomcat
> > > INFO: Loading instance registry:
> > > /etc/sysconfig/pki/tomcat/pki-tomcat/pki-tomcat
> > > INFO: Loading password config: /etc/pki/pki-tomcat/password.conf
> > > INFO: Loading subsystem: ca
> > > INFO: Loading subsystem config: /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
> > > INFO: Getting signing cert info for ca from CS.cfg
> > > INFO: Getting signing cert info for ca from NSS database
> > > INFO: Getting ocsp_signing cert info for ca from CS.cfg
> > > INFO: Getting ocsp_signing cert info for ca from NSS database
> > > INFO: Getting sslserver cert info for ca from CS.cfg
> > > INFO: Getting sslserver cert info for ca from NSS database
> > > INFO: Getting subsystem cert info for ca from CS.cfg
> > > INFO: Getting subsystem cert info for ca from NSS database
> > > INFO: Getting audit_signing cert info for ca from CS.cfg
> > > INFO: Getting audit_signing cert info for ca from NSS database
> > > INFO: Fixing the following certs: ['ca_ocsp_signing', 'sslserver',
> > > 'subsystem', 'ca_audit_signing']
> > > INFO: Stopping the instance to proceed with system cert renewal
> > > INFO: Selftests disabled for subsystems: ca
> > > INFO: Getting sslserver cert info for ca from CS.cfg
> > > INFO: Getting sslserver cert info for ca from NSS database
> > > INFO: Trying to create a new temp cert for sslserver.
> > > INFO: Generate temp SSL certificate
> > > INFO: Getting sslserver cert info for ca from CS.cfg
> > > INFO: Getting sslserver cert info for ca from NSS database
> > > INFO: CSR for sslserver has been written to
> > > /tmp/tmpg_738l5a/sslserver.csr
> > > INFO: Getting signing cert info for ca from CS.cfg
> > > INFO: Getting signing cert info for ca from NSS database
> > > INFO: CA cert written to /tmp/tmpg_738l5a/ca_certificate.crt
> > > INFO: AKI: 0x1D0F356A3E7A6968A231723231EB22DA5A01F542
> > > INFO: Temp cert for sslserver is available at
> > > /etc/pki/pki-tomcat/certs/sslserver.crt.
> > > INFO: Getting sslserver cert info for ca from CS.cfg
> > > INFO: Getting sslserver cert info for ca from NSS database
> > > INFO: Getting sslserver cert info for ca from CS.cfg
> > > INFO: Getting sslserver cert info for ca from NSS database
> > > INFO: Updating CS.cfg with the new certificate
> > > INFO: Getting ocsp_signing cert info for ca from CS.cfg
> > > INFO: Getting ocsp_signing cert info for ca from NSS database
> > > INFO: Trying to setup a secure connection to CA subsystem.
> > > INFO: Secure connection with CA is established.
> > > INFO: Placing cert creation request for serial: 49
> > > Traceback (most recent call last):
> > >    File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py",
> > > line 600, in urlopen
> > >      chunked=chunked)
> > >    File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py",
> > > line 343, in _make_request
> > >      self._validate_conn(conn)
> > >    File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py",
> > > line 849, in _validate_conn
> > >      conn.connect()
> > >    File "/usr/lib/python3.6/site-packages/urllib3/connection.py",
> > > line 356, in connect
> > >      ssl_context=context)
> > >    File "/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line
> > > 350, in ssl_wrap_socket
> > >      context.load_cert_chain(certfile, keyfile)
> > > ssl.SSLError: [X509: KEY_VALUES_MISMATCH] key values mismatch
> > > (_ssl.c:3550)
> > >
> > > During handling of the above exception, another exception occurred:
> > >
> > > Traceback (most recent call last):
> > >    File "/usr/lib/python3.6/site-packages/requests/adapters.py", line
> > > 449, in send
> > >      timeout=timeout
> > >    File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py",
> > > line 638, in urlopen
> > >      _stacktrace=sys.exc_info()[2])
> > >    File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py",
> > > line 398, in increment
> > >      raise MaxRetryError(_pool, url, error or ResponseError(cause))
> > > urllib3.exceptions.MaxRetryError:
> > > HTTPSConnectionPool(host='ipa2.chem.byu.edu', port=8443): Max retries
> > > exceeded with url: /ca/rest/certrequests/profiles/caManualRenewal
> > > (Caused by SSLError(SSLError(185073780, '[X509: KEY_VALUES_MISMATCH]
> > > key values mismatch (_ssl.c:3550)'),))
> > >
> > > During handling of the above exception, another exception occurred:
> > >
> > > Traceback (most recent call last):
> > >    File "/usr/lib/python3.6/site-packages/pki/server/pkiserver.py",
> > > line 119, in <module>
> > >      cli.execute(sys.argv)
> > >    File "/usr/lib/python3.6/site-packages/pki/server/pkiserver.py",
> > > line 111, in execute
> > >      super(PKIServerCLI, self).execute(args)
> > >    File "/usr/lib/python3.6/site-packages/pki/cli/__init__.py", line
> > > 204, in execute
> > >      module.execute(module_args)
> > >    File "/usr/lib/python3.6/site-packages/pki/cli/__init__.py", line
> > > 204, in execute
> > >      module.execute(module_args)
> > >    File "/usr/lib/python3.6/site-packages/pki/server/cli/cert.py",
> > > line 1154, in execute
> > >      renew=True)
> > >    File "/usr/lib/python3.6/site-packages/pki/server/__init__.py",
> > > line 1709, in cert_create
> > >      PKIServer.renew_certificate(connection, new_cert_file, serial)
> > >    File "/usr/lib/python3.6/site-packages/pki/server/__init__.py",
> > > line 202, in renew_certificate
> > >      ret = cert_client.enroll_cert(inputs=inputs,
> > > profile_id='caManualRenewal')
> > >    File "/usr/lib/python3.6/site-packages/pki/__init__.py", line 442,
> > > in handler
> > >      return fn_call(inst, *args, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/pki/cert.py", line 1011, in
> > > enroll_cert
> > >      enroll_request = self.create_enrollment_request(profile_id, inputs)
> > >    File "/usr/lib/python3.6/site-packages/pki/__init__.py", line 442,
> > > in handler
> > >      return fn_call(inst, *args, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/pki/cert.py", line 962, in
> > > create_enrollment_request
> > >      enrollment_template = self.get_enrollment_template(profile_id)
> > >    File "/usr/lib/python3.6/site-packages/pki/__init__.py", line 442,
> > > in handler
> > >      return fn_call(inst, *args, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/pki/cert.py", line 942, in
> > > get_enrollment_template
> > >      r = self.connection.get(url, self.headers)
> > >    File "/usr/lib/python3.6/site-packages/pki/client.py", line 46, in
> > > wrapper
> > >      return func(self, *args, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/pki/client.py", line 160,
> > > in get
> > >      timeout=timeout,
> > >    File "/usr/lib/python3.6/site-packages/requests/sessions.py", line
> > > 537, in get
> > >      return self.request('GET', url, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/requests/sessions.py", line
> > > 524, in request
> > >      resp = self.send(prep, **send_kwargs)
> > >    File "/usr/lib/python3.6/site-packages/requests/sessions.py", line
> > > 637, in send
> > >      r = adapter.send(request, **kwargs)
> > >    File "/usr/lib/python3.6/site-packages/requests/adapters.py", line
> > > 514, in send
> > >      raise SSLError(e, request=request)
> > > requests.exceptions.SSLError:
> > > HTTPSConnectionPool(host='ipa2.chem.byu.edu', port=8443): Max retries
> > > exceeded with url: /ca/rest/certrequests/profiles/caManualRenewal
> > > (Caused by SSLError(SSLError(185073780, '[X509: KEY_VALUES_MISMATCH]
> > > key values mismatch (_ssl.c:3550)'),))
> > > ERROR: HTTPSConnectionPool(host='ipa2.chem.byu.edu', port=8443): Max
> > > retries exceeded with url:
> > > /ca/rest/certrequests/profiles/caManualRenewal (Caused by
> > > SSLError(SSLError(185073780, '[X509: KEY_VALUES_MISMATCH] key values
> > > mismatch (_ssl.c:3550)'),))
> > >
> > > [root@ipa2 ~]# echo "--Certificate:" && openssl x509 -noout -modulus -in
> > > /var/lib/ipa/ra-agent.pem && echo "--Key:" && openssl rsa -noout
> > > -modulus -in /var/lib/ipa/ra-agent.key
> > > --Certificate:
> > > Modulus=CBA68178847F53CC0D4A45871FEA7CCFD879D6423923DF75E557011E4C458BD5207ACBB4EAFC7A460C050878621E22406E7661105F7D8E7AF448AA1F0988C908F3173B77D87581BE7006ED09F76F0D4D736050088E986133DE851A0FB93A101AA77921CC24063A94E5076FD74D4D81139E7A2AAC61B86D07D424158CE56913ABB08C1CDBDAE35CE2D7BCEB9EF9D5A909FFF502A02E1E847DC44E8672A8889A65A721DC78FE70C24D9E12BB4E88D7AD0A49948E2BF40A22431731872ADFC3B81E368C655EB9B00DC37D04526EE2917445D2FAEBCF116821486FF088E500F12128A8D90309E8ED275CDD48E77BDE0E3358A6A81BD9DD2AD5AC6F68315C96FC50E9
> > >
> > > --Key:
> > > Modulus=CBA68178847F53CC0D4A45871FEA7CCFD879D6423923DF75E557011E4C458BD5207ACBB4EAFC7A460C050878621E22406E7661105F7D8E7AF448AA1F0988C908F3173B77D87581BE7006ED09F76F0D4D736050088E986133DE851A0FB93A101AA77921CC24063A94E5076FD74D4D81139E7A2AAC61B86D07D424158CE56913ABB08C1CDBDAE35CE2D7BCEB9EF9D5A909FFF502A02E1E847DC44E8672A8889A65A721DC78FE70C24D9E12BB4E88D7AD0A49948E2BF40A22431731872ADFC3B81E368C655EB9B00DC37D04526EE2917445D2FAEBCF116821486FF088E500F12128A8D90309E8ED275CDD48E77BDE0E3358A6A81BD9DD2AD5AC6F68315C96FC50E9
> > >
> > > [root@ipa2 ~]# openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
> > > | openssl md5
> > > (stdin)= 0915781edbe620c5791cda50f310c538
> > > [root@ipa2 ~]# openssl x509 -noout -modulus -in
> > > /var/lib/ipa/ra-agent.pem | openssl md5
> > > (stdin)= 0915781edbe620c5791cda50f310c538
> > >
> > > Looking at the cert and the key, they are a match and modulus also
> > > matches.  What I can't figure out is why I am seeing this error if the
> > > key and cert match.  Is it possible to have a timestamp issue, or is
> > > there some other reason that I can't find.  Any help would be greatly
> > > appreciated.
> >
> > I'm not familiar with this command but based on the options you are
> > passing you compared the wrong cert. You compared the RA agent cert and
> > you asked to renew the subsystem cert.
> >
> > You might want to see what cert owns serial number 49.
> >
> > rob
>
> The reason these are the two compared is that there are no other keys on
> the server.  Looking through the documentation seems to indicate that
> all certs are generated from this key pair.  Is that not correct, and if
> it is not correct then where are the keys located for the other certs, I
> have been unable to locate them anywhere on the server.

The certs and keys are stored in the NSS database in
/etc/pki/pki-tomcat/alias/

rob