Hi,
I recently upgraded my FreeIPA host system from Fedorda 40 to 41. Since the upgrade I am unable to access the details of the CA subsystem.
While I get a list/overview of all certificates that are available in the directory, FreeIPA throws an error if I try to access a specific certificate or CA.
The error is:
IPA Error 907: Network Error cannot connect to 'https://my-idm-server.idm.my.domain:443/ca/rest/certs/2164020197888160700271...' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638)
I am also getting this error while running the ipa-healthcheck.
{ "source": "ipahealthcheck.dogtag.ca", "check": "DogtagCertsConnectivityCheck", "result": "ERROR", "uuid": "84949312-c4a1-4924-95e5-338894d2ee27", "when": "20250122094218Z", "duration": "0.022545", "kw": { "key": "cert_show_ra", "error": "cannot connect to 'https://my-idm-server.idm.my.domain:443/ca/rest/certs/1984213844249033578839...' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638)", "serial": "198421384424903357883919048254057663382", "msg": "Request for certificate failed: {error}" } }, [...]
I am able to get a working TLS Handshake and a sensible reply with curl on the same machine. At first I guessd it might be an incompatiblity with TLSv1.3, so I tried to configure only TLSv1.2 in the httpd ssl.conf, but this did not resolve the issue. I also tried to use the legacy system crypto-policy instead of the default one. So I don't really think that this is a cipher missmatch/compatiblity issue. Could this be a verification issue on the certificate chain somewhere?
Does someone maybe have a hint where to start looking next and get this fixed?
FreeIPA Version 4.12.2 OS: Fedora 41 Server, no upgrades pending, default repos.
Thank you for your help!
Cheers,
Hannes
Hi,
On Wed, Jan 22, 2025 at 10:56 AM Hannes Eberhardt via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Hi,
I recently upgraded my FreeIPA host system from Fedorda 40 to 41. Since the upgrade I am unable to access the details of the CA subsystem.
While I get a list/overview of all certificates that are available in the directory, FreeIPA throws an error if I try to access a specific certificate or CA.
The error is:
IPA Error 907: Network Error cannot connect to 'https:// my-idm-server.idm.my.domain:443/ca/rest/certs/2164020197888160700271539004937198265 ' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638)
I am also getting this error while running the ipa-healthcheck.
{ "source": "ipahealthcheck.dogtag.ca", "check": "DogtagCertsConnectivityCheck", "result": "ERROR", "uuid": "84949312-c4a1-4924-95e5-338894d2ee27", "when": "20250122094218Z", "duration": "0.022545", "kw": { "key": "cert_show_ra", "error": "cannot connect to ' https://my-idm-server.idm.my.domain:443/ca/rest/certs/1984213844249033578839... ' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638)", "serial": "198421384424903357883919048254057663382", "msg": "Request for certificate failed: {error}" } }, [...]
I am able to get a working TLS Handshake and a sensible reply with curl on the same machine. At first I guessd it might be an incompatiblity with TLSv1.3, so I tried to configure only TLSv1.2 in the httpd ssl.conf, but this did not resolve the issue. I also tried to use the legacy system crypto-policy instead of the default one. So I don't really think that this is a cipher missmatch/compatiblity issue. Could this be a verification issue on the certificate chain somewhere?
Does someone maybe have a hint where to start looking next and get this fixed?
I would start by checking that your certificates are not expired. What's the output of # getcert list executed on your server my-idm-server.idm.my.domain ? Check that all the certificates have "expires: " dates in the future.
flo
FreeIPA Version 4.12.2 OS: Fedora 41 Server, no upgrades pending, default repos.
Thank you for your help!
Cheers,
Hannes
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
I would start by checking that your certificates are not expired. What's the output of # getcert list executed on your server my-idm-server.idm.my.domain ? Check that all the certificates have "expires: " dates in the future.
flo
Hi Flo,
thank you for the hint. I just checked this, no certificate is expired. All of the certificates are expiring in 2026.
Request ID '20240717114825': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra- agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=[...] subject: CN=[...] issued: 2024-07-17 13:48:25 CEST expires: 2026-07-07 13:48:25 CEST key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-clientAuth profile: caSubsystemCert pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes [...]
Hannes
Hi,
you can enable debug logs for IPA with the following steps: - create a file /etc/ipa/server.conf containing the following lines: [global] debug = True
- restart ipa services with "ipactl restart" - try to use the command cert-show: kinit admin; ipa cert-show 198421384424903357883919048254057663382" I'm expecting the command to fail but it will generate useful logs.
Check in /var/log/pki/pki-tomcat/localhost_access_log.$DATE if you see lines similar to: 10.0.144.249 - ipara [23/Jan/2025:07:51:19 +0000] "GET /ca/rest/account/login HTTP/1.1" 200 198 10.0.144.249 - ipara [23/Jan/2025:07:51:19 +0000] "GET /ca/rest/authorities/32c7e520-9996-4b2b-8069-97de6842d79d/cert HTTP/1.1" 200 1158 10.0.144.249 - ipara [23/Jan/2025:07:51:19 +0000] "GET /ca/rest/account/logout HTTP/1.1" 204 -
If you have a line with "login" at the time you did the ipa cert-show command, it means that the command properly reached out to the tomcat server. No "login" line would point to a misconfiguration, probably in /etc/pki/pki-tomcat/server.xml or /etc/httpd/conf.d/ipa-pki-proxy.conf. The 443 port is configured in /etc/httpd/conf.d/ssl.conf but most of the calls directed to this port get proxied to an AJP connector handled by tomcat. If the "login" line is not visible it means this redirection is broken.
Check the logs in /var/logs/httpd/error_log, after the line that contains something similar to [Thu Jan 23 07:51:19.038273 2025] [wsgi:error] [pid 550528:tid 550860] [remote 10.0.144.249:41806] ipa: DEBUG: raw: cert_show('1', version='2.251') They may contain a bit more information about this "Network Error".
You also mentioned I am able to get a working TLS Handshake and a sensible reply with curl on the same machine. Which command did you run exactly?
Is the ipa-healthcheck error the only one?
flo
On Thu, Jan 23, 2025 at 12:01 AM Hannes Eberhardt via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
I would start by checking that your certificates are not expired. What's the output of # getcert list executed on your server my-idm-server.idm.my.domain ? Check that all the certificates have "expires: " dates in the future.
flo
Hi Flo,
thank you for the hint. I just checked this, no certificate is expired. All of the certificates are expiring in 2026.
Request ID '20240717114825': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra- agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=[...] subject: CN=[...] issued: 2024-07-17 13:48:25 CEST expires: 2026-07-07 13:48:25 CEST key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-clientAuth profile: caSubsystemCert pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes [...]
Hannes
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
I just check the logs you mentioned.
Check in /var/log/pki/pki-tomcat/localhost_access_log.$DATE
This is the access log after kinit and ipa-cert show:
10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "GET /ca/admin/ca/getStatus HTTP/1.1" 200 122 10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "GET /ca/admin/ca/getStatus HTTP/1.1" 200 122 10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "POST /ca/admin/ca/getStatus HTTP/1.1" 200 122 0:0:0:0:0:0:0:1 - - [23/Jan/2025:21:43:48 +0100] "-" 400 - 127.0.0.1 - - [23/Jan/2025:21:43:48 +0100] "-" 400 - 10.1.0.4 - - [23/Jan/2025:21:44:21 +0100] "GET /ca/rest/certs/198421384424903357883919048254057663382 HTTP/1.1" 200 12632
No /login line here.
Check the logs in /var/logs/httpd/error_log
There seems to be a verification error on the certificate.
[Thu Jan 23 21:44:21.532600 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: request GET https://%5B...%5D:443/ca/rest/certs/198421384424903357883919048254057663382 [Thu Jan 23 21:44:21.532617 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: request body '' [Thu Jan 23 21:44:21.537014 2025] [ssl:error] [pid 10413:tid 10547] [client 10.1.0.4:54670] AH02039: Certificate Verification: Error (50): application verification failure [Thu Jan 23 21:44:21.546571 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: httplib request failed: [Thu Jan 23 21:44:21.546589 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] Traceback (most recent call last): [Thu Jan 23 21:44:21.546592 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 272, in _httplib_request [Thu Jan 23 21:44:21.546595 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] res = conn.getresponse() [Thu Jan 23 21:44:21.546598 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 1428, in getresponse [Thu Jan 23 21:44:21.546601 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] response.begin() [Thu Jan 23 21:44:21.546603 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.546606 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 331, in begin [Thu Jan 23 21:44:21.546609 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] version, status, reason = self._read_status() [Thu Jan 23 21:44:21.546611 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.546614 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 292, in _read_status [Thu Jan 23 21:44:21.546616 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") [Thu Jan 23 21:44:21.546619 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546622 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/socket.py", line 719, in readinto [Thu Jan 23 21:44:21.546624 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sock.recv_into(b) [Thu Jan 23 21:44:21.546627 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^^^ [Thu Jan 23 21:44:21.546629 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1304, in recv_into [Thu Jan 23 21:44:21.546632 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.read(nbytes, buffer) [Thu Jan 23 21:44:21.546638 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546640 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1138, in read [Thu Jan 23 21:44:21.546643 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sslobj.read(len, buffer) [Thu Jan 23 21:44:21.546646 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546648 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638) [Thu Jan 23 21:44:21.564194 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: WSGI wsgi_execute PublicError: Traceback (most recent call last): [Thu Jan 23 21:44:21.564218 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 272, in _httplib_request [Thu Jan 23 21:44:21.564222 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] res = conn.getresponse() [Thu Jan 23 21:44:21.564225 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 1428, in getresponse [Thu Jan 23 21:44:21.564228 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] response.begin() [Thu Jan 23 21:44:21.564230 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.564233 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 331, in begin [Thu Jan 23 21:44:21.564236 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] version, status, reason = self._read_status() [Thu Jan 23 21:44:21.564238 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.564241 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 292, in _read_status [Thu Jan 23 21:44:21.564244 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") [Thu Jan 23 21:44:21.564246 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564249 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/socket.py", line 719, in readinto [Thu Jan 23 21:44:21.564252 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sock.recv_into(b) [Thu Jan 23 21:44:21.564254 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^^^ [Thu Jan 23 21:44:21.564257 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1304, in recv_into [Thu Jan 23 21:44:21.564260 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.read(nbytes, buffer) [Thu Jan 23 21:44:21.564262 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564265 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1138, in read [Thu Jan 23 21:44:21.564267 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sslobj.read(len, buffer) [Thu Jan 23 21:44:21.564270 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564272 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638) [Thu Jan 23 21:44:21.564275 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] [Thu Jan 23 21:44:21.564278 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] During handling of the above exception, another exception occurred: [Thu Jan 23 21:44:21.564284 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] [Thu Jan 23 21:44:21.564286 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] Traceback (most recent call last): [Thu Jan 23 21:44:21.564289 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/rpcserver.py", line 417, in wsgi_execute [Thu Jan 23 21:44:21.564292 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] result = command(*args, **options) [Thu Jan 23 21:44:21.564294 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 477, in __call__ [Thu Jan 23 21:44:21.564297 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.__do_call(*args, **options) [Thu Jan 23 21:44:21.564300 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564302 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 544, in __do_call [Thu Jan 23 21:44:21.564305 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ret = self.run(*args, **options) [Thu Jan 23 21:44:21.564308 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 885, in run [Thu Jan 23 21:44:21.564310 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.execute(*args, **options) [Thu Jan 23 21:44:21.564313 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564315 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/cert.py", line 1379, in execute [Thu Jan 23 21:44:21.564318 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] result = self.Backend.ra.get_certificate(serial_number) [Thu Jan 23 21:44:21.564321 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/dogtag.py", line 906, in get_certificate [Thu Jan 23 21:44:21.564324 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] _http_status, _http_headers, http_body = self._ssldo( [Thu Jan 23 21:44:21.564326 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~^ [Thu Jan 23 21:44:21.564329 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] 'GET', path, use_session=False, [Thu Jan 23 21:44:21.564331 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564334 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ...<2 lines>... [Thu Jan 23 21:44:21.564337 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] }, [Thu Jan 23 21:44:21.564339 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^ [Thu Jan 23 21:44:21.564342 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ) [Thu Jan 23 21:44:21.564344 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^ [Thu Jan 23 21:44:21.564347 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/dogtag.py", line 660, in _ssldo [Thu Jan 23 21:44:21.564350 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] status, resp_headers, resp_body = dogtag.https_request( [Thu Jan 23 21:44:21.564352 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^ [Thu Jan 23 21:44:21.564355 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] self.ca_host, self.override_port or self.env.ca_agent_port, [Thu Jan 23 21:44:21.564366 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564370 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ...<4 lines>... [Thu Jan 23 21:44:21.564372 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] method=method, headers=headers, body=body [Thu Jan 23 21:44:21.564375 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564377 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ) [Thu Jan 23 21:44:21.564380 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^ [Thu Jan 23 21:44:21.564382 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 216, in https_request [Thu Jan 23 21:44:21.564385 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return _httplib_request( [Thu Jan 23 21:44:21.564388 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] 'https', host, port, url, connection_factory, body, [Thu Jan 23 21:44:21.564390 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] method=method, headers=headers) [Thu Jan 23 21:44:21.564393 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 280, in _httplib_request [Thu Jan 23 21:44:21.564396 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] raise NetworkError(uri=uri, error=str(e)) [Thu Jan 23 21:44:21.564398 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipalib.errors.NetworkError: cannot connect to 'https://%5B...%5D:443/ca/rest/certs/198421384424903357883919048254057663382' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638)
You also mentioned I am able to get a working TLS Handshake and a sensible reply with curl on the same machine. Which command did you run exactly?
# curl -v https://%5B...%5D:443/ca/rest/certs/198421384424903357883919048254057663382 * Host [...]:443 was resolved. * IPv6: [...] * IPv4: [...] * Trying [...]:443... * Connected to [...] ([...]) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/pki/tls/certs/ca-bundle.crt * CApath: none * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS * ALPN: server accepted http/1.1 * Server certificate: * subject: [...] * start date: Jul 17 11:48:54 2024 GMT * expire date: Jul 18 11:48:54 2026 GMT * subjectAltName: host "[...]" matched cert's "[...]" * issuer: CN=onether.net IDM Intermediate CA * SSL certificate verify ok. * Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 1: Public key type RSA (3072/128 Bits/secBits), signed using sha256WithRSAEncryption * using HTTP/1.x
GET /ca/rest/certs/198421384424903357883919048254057663382 HTTP/1.1 Host: [...] User-Agent: curl/8.9.1 Accept: */*
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * Request completely sent off * TLSv1.3 (IN), TLS handshake, Request CERT (13): * TLSv1.3 (OUT), TLS handshake, Certificate (11): * TLSv1.3 (OUT), TLS handshake, Finished (20): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): < HTTP/1.1 200 200 < Date: Thu, 23 Jan 2025 20:58:05 GMT < Server: Apache/2.4.62 (Fedora Linux) OpenSSL/3.2.2 mod_wsgi/5.0.2 Python/3.13 mod_auth_gssapi/1.6.5 < Content-Type: application/json < Vary: Accept-Encoding < Transfer-Encoding: chunked < {"id":"0x9546918eec13583066bc7c638498cb96","IssuerDN":"CN=[...]
Is the ipa-healthcheck error the only one?
I have got nine of the ssl/tls alert handshake failures and this one additionally: { "source": "ipahealthcheck.ipa.certs", "check": "IPADogtagCertsMatchCheck", "result": "CRITICAL", "uuid": "67657ec3-d8ae-40b1-9e4c-4b7ded3e203b", "when": "20250123210034Z", "duration": "0.204734", "kw": { "exception": "no matching entry found", "traceback": "Traceback (most recent call last):\n File "/usr/lib/python3.13/site-packages/ipahealthcheck/core/core.py", line 56, in run_plugin\n for result in plugin.check():\n ~~~~~~~~~~~~^^\n File "/usr/lib/python3.13/site- packages/ipahealthcheck/core/plugin.py", line 18, in wrapper\n for result in f(*args, **kwds):\n ~^^^^^^^^^^^^^^^\n File "/usr/lib/python3.13/site-packages/ipahealthcheck/ipa/certs.py", line 950, in check\n ipaca_certs_ok = yield from match_ldap_nss_certs_by_subject(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<3 lines>...\n )\n ^\n File "/usr/lib/python3.13/site- packages/ipahealthcheck/ipa/certs.py", line 877, in match_ldap_nss_certs_by_subject\n entries = ldap.get_entries(\n dn,\n filter=f'subjectname={subject}'\n )\n File "/usr/lib/python3.13/site-packages/ipapython/ipaldap.py", line 1473, in get_entries\n entries, truncated = self.find_entries(\n ~~~~~~~~~~~~~~~~~^\n base_dn=base_dn, scope=scope, filter=filter, attrs_list=attrs_list,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n get_effective_rights=get_effective_rights,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n **kwargs)\n ^^^^^^^^^\n File "/usr/lib/python3.13/site- packages/ipapython/ipaldap.py", line 1617, in find_entries\n raise errors.EmptyResult(reason='no matching entry found')\nipalib.errors.EmptyResult: no matching entry found\n" } }
Then there are some deprecation warnings like: Properties that return a naïve datetime object have been deprecated. Please switch to not_valid_after_utc
Hannes
Hannes Eberhardt via FreeIPA-users wrote:
I just check the logs you mentioned.
Check in /var/log/pki/pki-tomcat/localhost_access_log.$DATE
This is the access log after kinit and ipa-cert show:
10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "GET /ca/admin/ca/getStatus HTTP/1.1" 200 122 10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "GET /ca/admin/ca/getStatus HTTP/1.1" 200 122 10.1.0.4 - - [23/Jan/2025:21:43:48 +0100] "POST /ca/admin/ca/getStatus HTTP/1.1" 200 122 0:0:0:0:0:0:0:1 - - [23/Jan/2025:21:43:48 +0100] "-" 400 - 127.0.0.1 - - [23/Jan/2025:21:43:48 +0100] "-" 400 - 10.1.0.4 - - [23/Jan/2025:21:44:21 +0100] "GET /ca/rest/certs/198421384424903357883919048254057663382 HTTP/1.1" 200 12632
No /login line here.
Check the logs in /var/logs/httpd/error_log
There seems to be a verification error on the certificate.
[Thu Jan 23 21:44:21.532600 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: request GET https://%5B...%5D:443/ca/rest/certs/198421384424903357883919048254057663382 [Thu Jan 23 21:44:21.532617 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: request body '' [Thu Jan 23 21:44:21.537014 2025] [ssl:error] [pid 10413:tid 10547] [client 10.1.0.4:54670] AH02039: Certificate Verification: Error (50): application verification failure [Thu Jan 23 21:44:21.546571 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: httplib request failed: [Thu Jan 23 21:44:21.546589 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] Traceback (most recent call last): [Thu Jan 23 21:44:21.546592 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 272, in _httplib_request [Thu Jan 23 21:44:21.546595 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] res = conn.getresponse() [Thu Jan 23 21:44:21.546598 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 1428, in getresponse [Thu Jan 23 21:44:21.546601 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] response.begin() [Thu Jan 23 21:44:21.546603 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.546606 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 331, in begin [Thu Jan 23 21:44:21.546609 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] version, status, reason = self._read_status() [Thu Jan 23 21:44:21.546611 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666]
[Thu Jan 23 21:44:21.546614 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 292, in _read_status [Thu Jan 23 21:44:21.546616 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") [Thu Jan 23 21:44:21.546619 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546622 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/socket.py", line 719, in readinto [Thu Jan 23 21:44:21.546624 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sock.recv_into(b) [Thu Jan 23 21:44:21.546627 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^^^ [Thu Jan 23 21:44:21.546629 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1304, in recv_into [Thu Jan 23 21:44:21.546632 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.read(nbytes, buffer) [Thu Jan 23 21:44:21.546638 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546640 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1138, in read [Thu Jan 23 21:44:21.546643 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sslobj.read(len, buffer) [Thu Jan 23 21:44:21.546646 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.546648 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638) [Thu Jan 23 21:44:21.564194 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipa: DEBUG: WSGI wsgi_execute PublicError: Traceback (most recent call last): [Thu Jan 23 21:44:21.564218 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 272, in _httplib_request [Thu Jan 23 21:44:21.564222 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] res = conn.getresponse() [Thu Jan 23 21:44:21.564225 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 1428, in getresponse [Thu Jan 23 21:44:21.564228 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] response.begin() [Thu Jan 23 21:44:21.564230 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.564233 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 331, in begin [Thu Jan 23 21:44:21.564236 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] version, status, reason = self._read_status() [Thu Jan 23 21:44:21.564238 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^ [Thu Jan 23 21:44:21.564241 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/http/client.py", line 292, in _read_status [Thu Jan 23 21:44:21.564244 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") [Thu Jan 23 21:44:21.564246 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564249 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/socket.py", line 719, in readinto [Thu Jan 23 21:44:21.564252 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sock.recv_into(b) [Thu Jan 23 21:44:21.564254 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^^^ [Thu Jan 23 21:44:21.564257 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1304, in recv_into [Thu Jan 23 21:44:21.564260 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.read(nbytes, buffer) [Thu Jan 23 21:44:21.564262 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564265 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib64/python3.13/ssl.py", line 1138, in read [Thu Jan 23 21:44:21.564267 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self._sslobj.read(len, buffer) [Thu Jan 23 21:44:21.564270 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564272 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638) [Thu Jan 23 21:44:21.564275 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] [Thu Jan 23 21:44:21.564278 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] During handling of the above exception, another exception occurred: [Thu Jan 23 21:44:21.564284 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] [Thu Jan 23 21:44:21.564286 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] Traceback (most recent call last): [Thu Jan 23 21:44:21.564289 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/rpcserver.py", line 417, in wsgi_execute [Thu Jan 23 21:44:21.564292 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] result = command(*args, **options) [Thu Jan 23 21:44:21.564294 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 477, in __call__ [Thu Jan 23 21:44:21.564297 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.__do_call(*args, **options) [Thu Jan 23 21:44:21.564300 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564302 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 544, in __do_call [Thu Jan 23 21:44:21.564305 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ret = self.run(*args, **options) [Thu Jan 23 21:44:21.564308 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipalib/frontend.py", line 885, in run [Thu Jan 23 21:44:21.564310 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return self.execute(*args, **options) [Thu Jan 23 21:44:21.564313 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564315 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/cert.py", line 1379, in execute [Thu Jan 23 21:44:21.564318 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] result = self.Backend.ra.get_certificate(serial_number) [Thu Jan 23 21:44:21.564321 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/dogtag.py", line 906, in get_certificate [Thu Jan 23 21:44:21.564324 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] _http_status, _http_headers, http_body = self._ssldo( [Thu Jan 23 21:44:21.564326 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~^ [Thu Jan 23 21:44:21.564329 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] 'GET', path, use_session=False, [Thu Jan 23 21:44:21.564331 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564334 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ...<2 lines>... [Thu Jan 23 21:44:21.564337 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] }, [Thu Jan 23 21:44:21.564339 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^ [Thu Jan 23 21:44:21.564342 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ) [Thu Jan 23 21:44:21.564344 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^ [Thu Jan 23 21:44:21.564347 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipaserver/plugins/dogtag.py", line 660, in _ssldo [Thu Jan 23 21:44:21.564350 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] status, resp_headers, resp_body = dogtag.https_request( [Thu Jan 23 21:44:21.564352 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ~~~~~~~~~~~~~~~~~~~~^ [Thu Jan 23 21:44:21.564355 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] self.ca_host, self.override_port or self.env.ca_agent_port, [Thu Jan 23 21:44:21.564366 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564370 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ...<4 lines>... [Thu Jan 23 21:44:21.564372 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] method=method, headers=headers, body=body [Thu Jan 23 21:44:21.564375 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [Thu Jan 23 21:44:21.564377 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ) [Thu Jan 23 21:44:21.564380 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ^ [Thu Jan 23 21:44:21.564382 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 216, in https_request [Thu Jan 23 21:44:21.564385 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] return _httplib_request( [Thu Jan 23 21:44:21.564388 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] 'https', host, port, url, connection_factory, body, [Thu Jan 23 21:44:21.564390 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] method=method, headers=headers) [Thu Jan 23 21:44:21.564393 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] File "/usr/lib/python3.13/site- packages/ipapython/dogtag.py", line 280, in _httplib_request [Thu Jan 23 21:44:21.564396 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] raise NetworkError(uri=uri, error=str(e)) [Thu Jan 23 21:44:21.564398 2025] [wsgi:error] [pid 10410:tid 10685] [remote 10.1.0.4:54666] ipalib.errors.NetworkError: cannot connect to 'https://[...]:443/ca/rest/certs/198421384424903357883919048254057663382' : [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:2638) > You also mentioned > I am able to get a working TLS Handshake and a sensible reply with > curl > on the same machine. > Which command did you run exactly? # curl -v https://[...]:443/ca/rest/certs/198421384424903357883919048254057663382 * Host [...]:443 was resolved. * IPv6: [...] * IPv4: [...] * Trying [...]:443... * Connected to [...] ([...]) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/pki/tls/certs/ca-bundle.crt * CApath: none * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS * ALPN: server accepted http/1.1 * Server certificate: * subject: [...] * start date: Jul 17 11:48:54 2024 GMT * expire date: Jul 18 11:48:54 2026 GMT * subjectAltName: host "[...]" matched cert's "[...]" * issuer: CN=onether.net IDM Intermediate CA * SSL certificate verify ok. * Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 1: Public key type RSA (3072/128 Bits/secBits), signed using sha256WithRSAEncryption * using HTTP/1.x > GET /ca/rest/certs/198421384424903357883919048254057663382 HTTP/1.1 > Host: [...] > User-Agent: curl/8.9.1 > Accept: */* > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * Request completely sent off * TLSv1.3 (IN), TLS handshake, Request CERT (13): * TLSv1.3 (OUT), TLS handshake, Certificate (11): * TLSv1.3 (OUT), TLS handshake, Finished (20): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): < HTTP/1.1 200 200 < Date: Thu, 23 Jan 2025 20:58:05 GMT < Server: Apache/2.4.62 (Fedora Linux) OpenSSL/3.2.2 mod_wsgi/5.0.2 Python/3.13 mod_auth_gssapi/1.6.5 < Content-Type: application/json < Vary: Accept-Encoding < Transfer-Encoding: chunked < {"id":"0x9546918eec13583066bc7c638498cb96","IssuerDN":"CN=[...]
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
Is the ipa-healthcheck error the only one?
I have got nine of the ssl/tls alert handshake failures and this one additionally: { "source": "ipahealthcheck.ipa.certs", "check": "IPADogtagCertsMatchCheck", "result": "CRITICAL", "uuid": "67657ec3-d8ae-40b1-9e4c-4b7ded3e203b", "when": "20250123210034Z", "duration": "0.204734", "kw": { "exception": "no matching entry found", "traceback": "Traceback (most recent call last):\n File "/usr/lib/python3.13/site-packages/ipahealthcheck/core/core.py", line 56, in run_plugin\n for result in plugin.check():\n
packages/ipahealthcheck/core/plugin.py\", line 18, in wrapper\n for result in f(*args, **kwds):\n ~^^^^^^^^^^^^^^^\n File \"/usr/lib/python3.13/site-packages/ipahealthcheck/ipa/certs.py\", line 950, in check\n ipaca_certs_ok = yield from match_ldap_nss_certs_by_subject(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<3 lines>...\n )\n ^\n File \"/usr/lib/python3.13/site- packages/ipahealthcheck/ipa/certs.py\", line 877, in match_ldap_nss_certs_by_subject\n entries = ldap.get_entries(\n dn,\n filter=f'subjectname={subject}'\n )\n File \"/usr/lib/python3.13/site-packages/ipapython/ipaldap.py\", line 1473, in get_entries\n entries, truncated = self.find_entries(\n ~~~~~~~~~~~~~~~~~^\n base_dn=base_dn, scope=scope, filter=filter, attrs_list=attrs_list,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n get_effective_rights=get_effective_rights,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n **kwargs)\n ^^^^^^^^^\n File \"/usr/lib/python3.13/site- packages/ipapython/ipaldap.py\", line 1617, in find_entries\n raise errors.EmptyResult(reason='no matching entry found')\nipalib.errors.EmptyResult: no matching entry found\n" } }
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
Then there are some deprecation warnings like: Properties that return a naïve datetime object have been deprecated. Please switch to not_valid_after_utc
Those deprecation warnings are partially fixed and will land in Fedora next week or so.
rob
Hi Rob,
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
No, I have neither replaced any of the internal certs nor the certficate chain.
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I've searched the slapd access log while running ipa-healtcheck for err=32. The log is very long but there are "only" four occurences of err=32. I think this should be the relevant lines that share the same op/operation(?) value. [...] [26/Jan/2025:12:56:33.608141106 +0100] conn=1847 op=10 SRCH base="cn=DNSSEC,cn=idm- [...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[...],dc=[...]" scope=0 filter="(objectClass=*)" attrs="cn" [26/Jan/2025:12:56:33.608208600 +0100] conn=1847 op=10 RESULT err=32 tag=101 nentries=0 wtime=0.000048841 optime=0.000068771 etime=0.000116566 [...] [26/Jan/2025:12:56:36.433609457 +0100] conn=1857 op=3 SRCH base="cn=changelog5,cn=config" scope=0 filter="(objectClass=*)" attrs="nsslapd-changelogmaxentries" [26/Jan/2025:12:56:36.433639656 +0100] conn=1857 op=3 RESULT err=32 tag=101 nentries=0 wtime=0.000173461 optime=0.000030915 etime=0.000203514 [...] [26/Jan/2025:12:56:36.434644495 +0100] conn=1847 op=18 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="* aci" [26/Jan/2025:12:56:36.434671918 +0100] conn=1847 op=18 RESULT err=32 tag=101 nentries=0 wtime=0.000090959 optime=0.000028427 etime=0.000118526 [...] [26/Jan/2025:12:57:14.989604635 +0100] conn=5 op=8240 SRCH base="cn=docarch.[...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[... ],dc=[...]" scope=0 filter="(objectClass=*)" attrs=ALL [26/Jan/2025:12:57:14.989643954 +0100] conn=5 op=8240 RESULT err=32 tag=101 nentries=0 wtime=0.000037066 optime=0.000039544 etime=0.000076147 [...]
If I did understand the logs right,there are a few objects missing: A DNSSEC cert, a changelog, something replica related and a service certificate I've signed by the CA.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
I can rule out the "someone cleaning up" part, but I can't rule out a database corruption. Is there maybe a way to check for this?
Hannes
Hannes Eberhardt wrote:
Hi Rob,
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
No, I have neither replaced any of the internal certs nor the certficate chain.
You could try creating /etc/ipa/server.conf with contents:
[global] debug = True
Then restart the httpd service.
You might get more or better error messages about the failure.
Note that the debugging is rather long winded so you may not to leave it running long.
You could also try re-exporting the cert chain.
# ipa-certupdate
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I've searched the slapd access log while running ipa-healtcheck for err=32. The log is very long but there are "only" four occurences of err=32. I think this should be the relevant lines that share the same op/operation(?) value. [...] [26/Jan/2025:12:56:33.608141106 +0100] conn=1847 op=10 SRCH base="cn=DNSSEC,cn=idm- [...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[...],dc=[...]" scope=0 filter="(objectClass=*)" attrs="cn" [26/Jan/2025:12:56:33.608208600 +0100] conn=1847 op=10 RESULT err=32 tag=101 nentries=0 wtime=0.000048841 optime=0.000068771 etime=0.000116566 [...] [26/Jan/2025:12:56:36.433609457 +0100] conn=1857 op=3 SRCH base="cn=changelog5,cn=config" scope=0 filter="(objectClass=*)" attrs="nsslapd-changelogmaxentries" [26/Jan/2025:12:56:36.433639656 +0100] conn=1857 op=3 RESULT err=32 tag=101 nentries=0 wtime=0.000173461 optime=0.000030915 etime=0.000203514 [...] [26/Jan/2025:12:56:36.434644495 +0100] conn=1847 op=18 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="* aci" [26/Jan/2025:12:56:36.434671918 +0100] conn=1847 op=18 RESULT err=32 tag=101 nentries=0 wtime=0.000090959 optime=0.000028427 etime=0.000118526 [...] [26/Jan/2025:12:57:14.989604635 +0100] conn=5 op=8240 SRCH base="cn=docarch.[...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[... ],dc=[...]" scope=0 filter="(objectClass=*)" attrs=ALL [26/Jan/2025:12:57:14.989643954 +0100] conn=5 op=8240 RESULT err=32 tag=101 nentries=0 wtime=0.000037066 optime=0.000039544 etime=0.000076147 [...]
If I did understand the logs right,there are a few objects missing: A DNSSEC cert, a changelog, something replica related and a service certificate I've signed by the CA.
Returning no entries not necessarily an error. In this case the backtrace showed a failure searching with filter like 'subjectname=<something>'. That isn't in any of your findings.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
I can rule out the "someone cleaning up" part, but I can't rule out a database corruption. Is there maybe a way to check for this?
Stop your DS instance.
# dsctl -v EXAMPLE-TEST dbverify userroot
rob
Hi,
after a long break I did find some time to get back to this problem and I do have some new findings.
After digging through the logs again, I did research on some error messages and found some hints that this could also be related to some OCSP check failing. In /etc/httpd/conf.d/ssl.conf I could find the "SSLOCSPEnable on" directive. After setting this to off, everything started working again and I was able to get a new replica up and running.
I did expect the new replica to misbehave in the same way but it didn't. There also is no "SSLOCSPEnable" directive in the ssl.conf of the replica. This brings me to the question, if there has been a change in FreeIPA at some point and maybe a config migration or something failed.
Do you know if the current correct state of httpd config is an enabled or disabled OCSP verification?
Thank you and best regards Hannes
On Tue, 2025-01-28 at 13:59 -0500, Rob Crittenden via FreeIPA-users wrote:
Hannes Eberhardt wrote:
Hi Rob,
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
No, I have neither replaced any of the internal certs nor the certficate chain.
You could try creating /etc/ipa/server.conf with contents:
[global] debug = True
Then restart the httpd service.
You might get more or better error messages about the failure.
Note that the debugging is rather long winded so you may not to leave it running long.
You could also try re-exporting the cert chain.
# ipa-certupdate
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I've searched the slapd access log while running ipa-healtcheck for err=32. The log is very long but there are "only" four occurences of err=32. I think this should be the relevant lines that share the same op/operation(?) value. [...] [26/Jan/2025:12:56:33.608141106 +0100] conn=1847 op=10 SRCH base="cn=DNSSEC,cn=idm- [...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[...],dc=[...]" scope=0 filter="(objectClass=*)" attrs="cn" [26/Jan/2025:12:56:33.608208600 +0100] conn=1847 op=10 RESULT err=32 tag=101 nentries=0 wtime=0.000048841 optime=0.000068771 etime=0.000116566 [...] [26/Jan/2025:12:56:36.433609457 +0100] conn=1857 op=3 SRCH base="cn=changelog5,cn=config" scope=0 filter="(objectClass=*)" attrs="nsslapd-changelogmaxentries" [26/Jan/2025:12:56:36.433639656 +0100] conn=1857 op=3 RESULT err=32 tag=101 nentries=0 wtime=0.000173461 optime=0.000030915 etime=0.000203514 [...] [26/Jan/2025:12:56:36.434644495 +0100] conn=1847 op=18 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="* aci" [26/Jan/2025:12:56:36.434671918 +0100] conn=1847 op=18 RESULT err=32 tag=101 nentries=0 wtime=0.000090959 optime=0.000028427 etime=0.000118526 [...] [26/Jan/2025:12:57:14.989604635 +0100] conn=5 op=8240 SRCH base="cn=docarch.[...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc= [... ],dc=[...]" scope=0 filter="(objectClass=*)" attrs=ALL [26/Jan/2025:12:57:14.989643954 +0100] conn=5 op=8240 RESULT err=32 tag=101 nentries=0 wtime=0.000037066 optime=0.000039544 etime=0.000076147 [...]
If I did understand the logs right,there are a few objects missing: A DNSSEC cert, a changelog, something replica related and a service certificate I've signed by the CA.
Returning no entries not necessarily an error. In this case the backtrace showed a failure searching with filter like 'subjectname=<something>'. That isn't in any of your findings.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
I can rule out the "someone cleaning up" part, but I can't rule out a database corruption. Is there maybe a way to check for this?
Stop your DS instance.
# dsctl -v EXAMPLE-TEST dbverify userroot
rob
Hi,
On Mon, Jun 30, 2025 at 3:22 PM Hannes Eberhardt via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Hi,
after a long break I did find some time to get back to this problem and I do have some new findings.
After digging through the logs again, I did research on some error messages and found some hints that this could also be related to some OCSP check failing. In /etc/httpd/conf.d/ssl.conf I could find the "SSLOCSPEnable on" directive. After setting this to off, everything started working again and I was able to get a new replica up and running.
IIRC this setting is enabled when the server is configured for smart card authentication. flo
I did expect the new replica to misbehave in the same way but it didn't. There also is no "SSLOCSPEnable" directive in the ssl.conf of the replica. This brings me to the question, if there has been a change in FreeIPA at some point and maybe a config migration or something failed.
Do you know if the current correct state of httpd config is an enabled or disabled OCSP verification?
Thank you and best regards Hannes
On Tue, 2025-01-28 at 13:59 -0500, Rob Crittenden via FreeIPA-users wrote:
Hannes Eberhardt wrote:
Hi Rob,
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
No, I have neither replaced any of the internal certs nor the certficate chain.
You could try creating /etc/ipa/server.conf with contents:
[global] debug = True
Then restart the httpd service.
You might get more or better error messages about the failure.
Note that the debugging is rather long winded so you may not to leave it running long.
You could also try re-exporting the cert chain.
# ipa-certupdate
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I've searched the slapd access log while running ipa-healtcheck for err=32. The log is very long but there are "only" four occurences of err=32. I think this should be the relevant lines that share the same op/operation(?) value. [...] [26/Jan/2025:12:56:33.608141106 +0100] conn=1847 op=10 SRCH base="cn=DNSSEC,cn=idm- [...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[...],dc=[...]" scope=0 filter="(objectClass=*)" attrs="cn" [26/Jan/2025:12:56:33.608208600 +0100] conn=1847 op=10 RESULT err=32 tag=101 nentries=0 wtime=0.000048841 optime=0.000068771 etime=0.000116566 [...] [26/Jan/2025:12:56:36.433609457 +0100] conn=1857 op=3 SRCH base="cn=changelog5,cn=config" scope=0 filter="(objectClass=*)" attrs="nsslapd-changelogmaxentries" [26/Jan/2025:12:56:36.433639656 +0100] conn=1857 op=3 RESULT err=32 tag=101 nentries=0 wtime=0.000173461 optime=0.000030915 etime=0.000203514 [...] [26/Jan/2025:12:56:36.434644495 +0100] conn=1847 op=18 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="* aci" [26/Jan/2025:12:56:36.434671918 +0100] conn=1847 op=18 RESULT err=32 tag=101 nentries=0 wtime=0.000090959 optime=0.000028427 etime=0.000118526 [...] [26/Jan/2025:12:57:14.989604635 +0100] conn=5 op=8240 SRCH base="cn=docarch.[...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc= [... ],dc=[...]" scope=0 filter="(objectClass=*)" attrs=ALL [26/Jan/2025:12:57:14.989643954 +0100] conn=5 op=8240 RESULT err=32 tag=101 nentries=0 wtime=0.000037066 optime=0.000039544 etime=0.000076147 [...]
If I did understand the logs right,there are a few objects missing: A DNSSEC cert, a changelog, something replica related and a service certificate I've signed by the CA.
Returning no entries not necessarily an error. In this case the backtrace showed a failure searching with filter like 'subjectname=<something>'. That isn't in any of your findings.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
I can rule out the "someone cleaning up" part, but I can't rule out a database corruption. Is there maybe a way to check for this?
Stop your DS instance.
# dsctl -v EXAMPLE-TEST dbverify userroot
rob
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Hannes Eberhardt via FreeIPA-users wrote:
Hi,
after a long break I did find some time to get back to this problem and I do have some new findings.
After digging through the logs again, I did research on some error messages and found some hints that this could also be related to some OCSP check failing. In /etc/httpd/conf.d/ssl.conf I could find the "SSLOCSPEnable on" directive. After setting this to off, everything started working again and I was able to get a new replica up and running.
I did expect the new replica to misbehave in the same way but it didn't. There also is no "SSLOCSPEnable" directive in the ssl.conf of the replica. This brings me to the question, if there has been a change in FreeIPA at some point and maybe a config migration or something failed.
Do you know if the current correct state of httpd config is an enabled or disabled OCSP verification?
IPA disables OCSP during installation. This is why on your replica it is disabled.
The only place that IPA enables OCSP is in ipa-advise when configuring smart cards.
rob
On Tue, 2025-01-28 at 13:59 -0500, Rob Crittenden via FreeIPA-users wrote:
Hannes Eberhardt wrote:
Hi Rob,
Have you recently replaced the CA chain and/or the IPA server cert(s)? Apache and/or DS?
No, I have neither replaced any of the internal certs nor the certficate chain.
You could try creating /etc/ipa/server.conf with contents:
[global] debug = True
Then restart the httpd service.
You might get more or better error messages about the failure.
Note that the debugging is rather long winded so you may not to leave it running long.
You could also try re-exporting the cert chain.
# ipa-certupdate
This means that one of the CA subsystem certificates was not found by the CA which is unexpected, hence the backtrace. You can try running healthcheck again and then watching the DS access log to find the query that returned nothing (err=32). That will tell you which subject it couldn't find.
I've searched the slapd access log while running ipa-healtcheck for err=32. The log is very long but there are "only" four occurences of err=32. I think this should be the relevant lines that share the same op/operation(?) value. [...] [26/Jan/2025:12:56:33.608141106 +0100] conn=1847 op=10 SRCH base="cn=DNSSEC,cn=idm- [...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc=[...],dc=[...]" scope=0 filter="(objectClass=*)" attrs="cn" [26/Jan/2025:12:56:33.608208600 +0100] conn=1847 op=10 RESULT err=32 tag=101 nentries=0 wtime=0.000048841 optime=0.000068771 etime=0.000116566 [...] [26/Jan/2025:12:56:36.433609457 +0100] conn=1857 op=3 SRCH base="cn=changelog5,cn=config" scope=0 filter="(objectClass=*)" attrs="nsslapd-changelogmaxentries" [26/Jan/2025:12:56:36.433639656 +0100] conn=1857 op=3 RESULT err=32 tag=101 nentries=0 wtime=0.000173461 optime=0.000030915 etime=0.000203514 [...] [26/Jan/2025:12:56:36.434644495 +0100] conn=1847 op=18 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="* aci" [26/Jan/2025:12:56:36.434671918 +0100] conn=1847 op=18 RESULT err=32 tag=101 nentries=0 wtime=0.000090959 optime=0.000028427 etime=0.000118526 [...] [26/Jan/2025:12:57:14.989604635 +0100] conn=5 op=8240 SRCH base="cn=docarch.[...],cn=masters,cn=ipa,cn=etc,dc=idm,dc=[...],dc= [... ],dc=[...]" scope=0 filter="(objectClass=*)" attrs=ALL [26/Jan/2025:12:57:14.989643954 +0100] conn=5 op=8240 RESULT err=32 tag=101 nentries=0 wtime=0.000037066 optime=0.000039544 etime=0.000076147 [...]
If I did understand the logs right,there are a few objects missing: A DNSSEC cert, a changelog, something replica related and a service certificate I've signed by the CA.
Returning no entries not necessarily an error. In this case the backtrace showed a failure searching with filter like 'subjectname=<something>'. That isn't in any of your findings.
I can't explain why a certificate would go missing. Did you have any recent db corruption? Did anyone attempt to "clean up" some records?
I can rule out the "someone cleaning up" part, but I can't rule out a database corruption. Is there maybe a way to check for this?
Stop your DS instance.
# dsctl -v EXAMPLE-TEST dbverify userroot
rob
freeipa-users@lists.fedorahosted.org
-
Florence Blanc-Renaud -
Hannes Eberhardt -
Rob Crittenden