[Freeipa-users] Re: freeipa expired certificates

[Rob Crittenden]

john john via FreeIPA-users wrote:
> Hello,
>
> I have a freeipa server (ipa-server-4.5.0-22.el7.centos.x86_64).
> Сertificates expired in April 2022 and why certmonger did not renew them is not clear.
>
> getcert list
>
> Request ID '20180510155654':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
>         certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=IPA RA,O=EXAMPLE.COM
>         expires: 2024-03-07 17:47:25 UTC
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
>         post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
>         track: yes
>         auto-renew: yes
> Request ID '20180510155804':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
>         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=CA Audit,O=EXAMPLE.COM
>         expires: 2024-03-05 17:47:13 UTC
>         key usage: digitalSignature,nonRepudiation
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20180510155805':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
>         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=OCSP Subsystem,O=EXAMPLE.COM
>         expires: 2024-03-07 17:47:15 UTC
>         eku: id-kp-OCSPSigning
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20180510155806':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
>         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=CA Subsystem,O=EXAMPLE.COM
>         expires: 2024-03-05 17:47:23 UTC
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20180510155807':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
>         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=Certificate Authority,O=EXAMPLE.COM
>         expires: 2038-05-10 15:56:32 UTC
>         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20180510155808':
>         status: CA_UNREACHABLE
>         ca-error: Error 60 connecting to https://freeipa.example.com:8443/ca/agent/ca/profileReview: Peer certificate cannot be authenticated with given CA certificates.
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
>         certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-04-15 04:47:25 UTC
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20180510155834':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Failed connect to freeipa.example.com:443; Connection refused).
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-EXAMPLE-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-EXAMPLE-COM/pwdfile.txt'
>         certificate: type=NSSDB,location='/etc/dirsrv/slapd-EXAMPLE-COM',nickname='Server-Cert',token='NSS Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-04-25 20:55:59 UTC
>         dns: freeipa.example.com
>         principal name: ldap/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv EXAMPLE-COM
>         track: yes
>         auto-renew: yes
> Request ID '20180510155907':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Failed connect to freeipa.example.com:443; Connection refused).
>         stuck: no
>         key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>         certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-04-26 06:11:51 UTC
>         dns: freeipa.example.com
>         principal name: ldap/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>         track: yes
>         auto-renew: yes
> Request ID '20180510155922':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Failed connect to freeipa.example.com:443; Connection refused).
>         stuck: no
>         key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>         certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-04-25 20:56:54 UTC
>         principal name: krbtgt/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-pkinit-KPKdc
>         pre-save command: 
>         post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
>         track: yes
>         auto-renew: yes
> Request ID '20180720144614':
>         status: CA_REJECTED
>         ca-error: Server at https://freeipa.example.com/ipa/xml denied our request, giving up: 2100 (RPC failed at server.  Insufficient access: Insufficient 'add' privilege to add the entry 'krbprincipalname=HTTP/[e-mail],cn=services,cn=accounts,dc=example,dc=cb,dc=com'.).
>         stuck: yes
>         key pair storage: type=FILE,location='/etc/pki/tls/private/pb-freeipa.key'
>         certificate: type=FILE,location='/etc/pki/tls/certs/pb-freeipa.crt'
>         CA: IPA
>         issuer: 
>         subject: 
>         expires: unknown
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20180720151813':
>         status: NEED_KEY_GEN_PIN
>         stuck: yes
>         key pair storage: type=NSSDB,location='/etc/ipa/certdb',nickname='Server-Cert',pin set
>         certificate: type=NSSDB,location='/etc/ipa/certdb',nickname='Server-Cert'
>         CA: IPA
>         issuer: 
>         subject: 
>         expires: unknown
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20180720152853':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Failed connect to freeipa.example.com:443; Connection refused).
>         stuck: no
>         key pair storage: type=FILE,location='/etc/pki/tls/private/freeipa.example.com.key'
>         certificate: type=FILE,location='/etc/pki/tls/certs/freeipa.example.com.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-04-25 20:57:24 UTC
>         dns: freeipa.example.com
>         principal name: HTTP/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20180723075009':
>         status: NEED_CSR
>         stuck: no
>         key pair storage: type=FILE,location='/root/OVPN_CLIENT_1.key'
>         certificate: type=FILE,location='/root/OVPN_CLIENT_1.pem'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2020-07-23 07:50:10 UTC
>         dns: freeipa.example.com
>         principal name: host/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20180723075356':
>         status: CA_REJECTED
>         ca-error: Server at https://freeipa.example.com/ipa/xml denied our request, giving up: 3009 (RPC failed at server.  invalid 'csr': hostname in subject of request 'OVPN_CLIENT_1' does not match name or aliases of principal 'HTTP/[e-mail]').
>         stuck: yes
>         key pair storage: type=FILE,location='/root/OVPN_CLIENT_2.key'
>         certificate: type=FILE,location='/root/OVPN_CLIENT_2.pem'
>         CA: IPA
>         issuer: 
>         subject: 
>         expires: unknown
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20180723075553':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Peer's Certificate has expired.).
>         stuck: no
>         key pair storage: type=FILE,location='/root/OVPN_CLIENT_3.key'
>         certificate: type=FILE,location='/root/OVPN_CLIENT_3.pem'
>         CA: IPA
>         issuer: 
>         subject: 
>         expires: unknown
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20200514145151':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Peer's Certificate has expired.).
>         stuck: no
>         key pair storage: type=FILE,location='/home/user/vpn-user.key'
>         certificate: type=FILE,location='/home/user/vpn-user.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-05-15 14:51:52 UTC
>         dns: freeipa.example.com
>         principal name: host/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
> Request ID '20200514150206':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://freeipa.example.com/ipa/xml failed request, will retry: -504 (libcurl failed to execute the HTTP POST transaction, explaining:  Peer's Certificate has expired.).
>         stuck: no
>         key pair storage: type=FILE,location='/home/user/freeipa.example.com.key'
>         certificate: type=FILE,location='/home/user/freeipa.example.com.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=EXAMPLE.COM
>         subject: CN=freeipa.example.com,O=EXAMPLE.COM
>         expires: 2022-05-15 15:02:07 UTC
>         dns: freeipa.example.com
>         principal name: host/[e-mail]
>         key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: 
>         post-save command: 
>         track: yes
>         auto-renew: yes
>
> I tried to update the certificates using the information from the following links:
>
> https://floblanc.wordpress.com/2016/12/06/using-certmonger-to-track-certific...
> https://floblanc.wordpress.com/2016/12/19/troubleshooting-certmonger-issues-...
> https://listman.redhat.com/archives/freeipa-users/2017-January/msg00216.html
>
> but it was not possible to update expired certificates.

So some of the certificates were being renewed in early March, looks
like as expected, but then something went sideways and the CA would no
longer start and the others just failed.

I'd suggest:

ipactl stop
make sure ntpd/chronyd is stopped
set date to March 8 (all certs should be valid then)
manually start the IPA services: dirsrv, krb5kdc, named if configured,
httpd, pki-tomcatd

At this point most everything should be running.

You can either restart certmonger and let it notice the expiring certs
and watch it to see that the certs are renewed.

Or manually run: getcert resubmit -i <id> -w -v to be able to more
easily watch each one install. For the CA-related certs give it some
time post renewal for the service to restart.

Then stop all the services again, return to today, ipactl start.

rob