Travis West via FreeIPA-users wrote:
I've restored the Renewal Master from before I started changing this. If I run getcert list I do see 9 certificates being tracked. None of the system certs seem to expire at the same time, but they also all have incorrect Common Name in the Subject. The RA cert is also expired and has an incorrect Common Name in the Subject
# getcert list Number of certificates and requests being tracked: 9. Request ID '20190322031541': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-IPA-****-NET',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-IPA-****-NET/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-IPA-****-NET',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=ipa1-sea2.ipa.****.net,O=IPA.****.NET expires: 2025-01-26 11:37:18 UTC dns: ipa1-sea2.ipa.****.net principal name: ldap/ipa1-sea2.ipa.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv IPA-****-NET track: yes auto-renew: yes Request ID '20190322031615': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=ipa1-sea2.ipa.****.net,O=IPA.****.NET expires: 2025-01-26 11:37:04 UTC dns: ipa1-sea2.ipa.****.net principal name: HTTP/ipa1-sea2.ipa.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20190322032004': status: CA_UNREACHABLE ca-error: Internal error stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=iso1.sea2.****.net,O=IPA.****.NET expires: 2021-03-08 03:28:16 UTC dns: iso1.sea2.****.net principal name: HOST/iso1.sea2.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes Request ID '20190322032029': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=mbc-hv1.sea2.****.net,O=IPA.****.NET expires: 2026-02-10 23:07:57 UTC dns: mbc-hv1.sea2.****.net principal name: HOST/mbc-hv1.sea2.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20190322032030': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=vault-backup2.sea2.****.net,O=IPA.****.NET expires: 2026-02-10 23:08:07 UTC dns: vault-backup2.sea2.****.net principal name: HOST/vault-backup2.sea2.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20190322032031': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=vault-hv1.sea2.****.net,O=IPA.****.NET expires: 2021-03-08 04:56:05 UTC dns: vault-hv1.sea2.****.net principal name: HOST/vault-hv1.sea2.****.net@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes Request ID '20190322032032': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=Certificate Authority,O=IPA.****.NET expires: 2037-03-21 04:43:44 UTC key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20190322032033': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=ipa1-sea2.ipa.****.net,O=IPA.****.NET expires: 2024-12-24 11:37:06 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca" track: yes auto-renew: yes Request ID '20190322032117': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key' certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt' CA: IPA issuer: CN=Certificate Authority,O=IPA.****.NET subject: CN=ipa1-sea2.ipa.****.net,O=IPA.****.NET expires: 2025-01-26 11:41:35 UTC principal name: krbtgt/IPA.****.NET@IPA.****.NET key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-pkinit-KPKdc pre-save command: post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert track: yes auto-renew: yes
I think the root cause is the certmonger tracking has been modified. There should be no principal associated with any request other than HTTP, ldap and pkint. The host principal is associated with most of the CA-related certs. I've never seen this before. It is also possible whoever messed with the tracking added -N to set the subject to CN=$FQDN. That would explain the bad subjects.
There is likely no going back in time for this one as the cert validity ranges are from 2021-2026 somehow. certmonger doesn't report on issued time so you'd have to look at each cert using certutil to see if there is a time where all the certs are still valid and see if the CA will start.
I'm just not sure how to get out of this pickle. Since the subject Common Name of the certificates is incorrect, I don't think setting the time back will solve this. I could potentially do an IPA data only backup (my understanding is that this doesn't include system certs). Then reinstall each of the 6 servers, install IPA again and restore the data backup. I believe there may be problems with this method as the /etc/ipa/ca.crt will likely change which I believe would affect the 389 hosts that use IPA.
The Kerberos keys would change which would mean a non-functional system after the restore. Plus all the clients.
rob