SSL errors ... again

[Justin Sanderson]

Ok. So once again my IPA server is having cert issues. Everything seems 
to be working except when I am in the web interface and goto 
"Authentication" --> "Certificates" --> Click any of the certs
in the list.


---- I get this error from the browser.------

IPA ERROR 907: NetworkError

cannot connect to 
https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' : 
SSL_HANDSHAKE_FAILURE


# getcert list |grep expires  --> everything checks out ok. no expiry on 
any of the certs


--- checked all the certs on there "Not Before" and "Not After" dates

for the following NSS db's

certutil -L -d /etc/pki/pki-tomcat/alias

certutil -L -d /etc/httpd/alias



  ---- In /var/log/httpd/error_log, I do see some errors: ----

Bad Remote Server Certificate -8181

SSL Library Error: -8181 Certificate has expired


I know it's an expired cert obviously from httpd errorlog but where is 
the darn thing. I thought i checked all the places and looked ok but I'm 
definitely missing something....


could use some advice.


TIA

[Rob Crittenden]

Justin Sanderson via FreeIPA-users wrote:
>  
>  Ok. So once again my IPA server is having cert issues. Everything seems
>  to be working except when I am in the web interface and goto
>  "Authentication" --> "Certificates" --> Click any of the certs
> in the list.
>  
>  
>  ---- I get this error from the browser.------
>  
>  IPA ERROR 907: NetworkError
>  
>  cannot connect to
>  https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' :
>  SSL_HANDSHAKE_FAILURE
>  
>  
>  # getcert list |grep expires  --> everything checks out ok. no expiry on
>  any of the certs
>  
>  
>  --- checked all the certs on there "Not Before" and "Not After"
> dates
>  for the following NSS db's
>  
>  certutil -L -d /etc/pki/pki-tomcat/alias
>  
>  certutil -L -d /etc/httpd/alias
>  
>  
>  
>   ---- In /var/log/httpd/error_log, I do see some errors: ----
>  
>  Bad Remote Server Certificate -8181
>  
>  SSL Library Error: -8181 Certificate has expired
>  
>  
>  I know it's an expired cert obviously from httpd errorlog but where is
>  the darn thing. I thought i checked all the places and looked ok but I'm
>  definitely missing something....
>  
>  
>  could use some advice. 
I'd simplify by trying on the command line: ipa cert-show 1

This will exercise the basic connectivity and will be less noisy than
using the UI. I'd run the same command on all servers you have in case
only one is affected.

As for the TLS error in the httpd.log its hard to say without broader
context. Is there an access log entry at the same time which may correlate?

rob

[Justin Sanderson]

I tried the "ipa cert-show 1" from the CLI and got the same error:

https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' :
SSL_HANDSHAKE_FAILURE


I do have a corresponding entry in the access_log for apache

"POST /ca/agent/ca/displayBySerial HTTP/1.1" 403 229


The apache error_log just re-iterates the same error as before which is 
the same time the access_log is updated:

Bad Remote Server Certificate -8181

SSL Library Error: -8181 Certificate has expired


Still looking....




On 5/7/2023 10:08 AM, Rob Crittenden wrote:
> Justin Sanderson via FreeIPA-users wrote:
>> Ok. So once again my IPA server is having cert issues. Everything seems
>> to be working except when I am in the web interface and goto
>> "Authentication" --> "Certificates" --> Click any of
the certs in the list.
>>
>>
>> ---- I get this error from the browser.------
>>
>> IPA ERROR 907: NetworkError
>>
>> cannot connect to
>> https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' :
>> SSL_HANDSHAKE_FAILURE
>>
>>
>> # getcert list |grep expires  --> everything checks out ok. no expiry on
>> any of the certs
>>
>>
>> --- checked all the certs on there "Not Before" and "Not
After" dates
>> for the following NSS db's
>>
>> certutil -L -d /etc/pki/pki-tomcat/alias
>>
>> certutil -L -d /etc/httpd/alias
>>
>>
>>
>>   ---- In /var/log/httpd/error_log, I do see some errors: ----
>>
>> Bad Remote Server Certificate -8181
>>
>> SSL Library Error: -8181 Certificate has expired
>>
>>
>> I know it's an expired cert obviously from httpd errorlog but where is
>> the darn thing. I thought i checked all the places and looked ok but I'm
>> definitely missing something....
>>
>>
>> could use some advice.
> I'd simplify by trying on the command line: ipa cert-show 1
>
> This will exercise the basic connectivity and will be less noisy than
> using the UI. I'd run the same command on all servers you have in case
> only one is affected.
>
> As for the TLS error in the httpd.log its hard to say without broader
> context. Is there an access log entry at the same time which may correlate?
>
> rob
>

[Justin Sanderson]

Found the culprit.... /var/lib/ipa/ra-agent.pem


# openssl -in /var/lib/ipa/ra-agent.pem -noout -text |grep "Not After"

The cert expired 4 days ago. ... whats proper "IPA" way to recreate 
cert. I could do it with openssl but idd if there's "hooks" to other 
components that i need to update.


On 5/7/2023 10:08 AM, Rob Crittenden wrote:
> Justin Sanderson via FreeIPA-users wrote:
>> Ok. So once again my IPA server is having cert issues. Everything seems
>> to be working except when I am in the web interface and goto
>> "Authentication" --> "Certificates" --> Click any of
the certs in the list.
>>
>>
>> ---- I get this error from the browser.------
>>
>> IPA ERROR 907: NetworkError
>>
>> cannot connect to
>> https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' :
>> SSL_HANDSHAKE_FAILURE
>>
>>
>> # getcert list |grep expires  --> everything checks out ok. no expiry on
>> any of the certs
>>
>>
>> --- checked all the certs on there "Not Before" and "Not
After" dates
>> for the following NSS db's
>>
>> certutil -L -d /etc/pki/pki-tomcat/alias
>>
>> certutil -L -d /etc/httpd/alias
>>
>>
>>
>>   ---- In /var/log/httpd/error_log, I do see some errors: ----
>>
>> Bad Remote Server Certificate -8181
>>
>> SSL Library Error: -8181 Certificate has expired
>>
>>
>> I know it's an expired cert obviously from httpd errorlog but where is
>> the darn thing. I thought i checked all the places and looked ok but I'm
>> definitely missing something....
>>
>>
>> could use some advice.
> I'd simplify by trying on the command line: ipa cert-show 1
>
> This will exercise the basic connectivity and will be less noisy than
> using the UI. I'd run the same command on all servers you have in case
> only one is affected.
>
> As for the TLS error in the httpd.log its hard to say without broader
> context. Is there an access log entry at the same time which may correlate?
>
> rob
>

[Florence Blanc-Renaud]

Hi Justin,

The ra-agent.pem is the same certificate on all servers/replicas. When
everything works properly, it gets renewed on the renewal master, then it
is uploaded in LDAP and the other replicas can download it from LDAP.
Do you have multiple servers? If yes and if the ra-agent.pem has been
renewed on another server, you can simply copy the ra-agent.pem file from
the other server to the failing one.
If you have multiple servers but the ra-agent.pem is expired on all of
them, you will have to fix the renewal master first. To find which server
is the renewal master:
# *kinit admin*
Password for admin(a)IPA.TEST:
# *ipa config-show | grep renewal*
  IPA CA renewal master: server.ipa.test
#

Then to fix the renewal master, you can use *ipa-cert-fix* command.

HTH,
flo

On Tue, May 9, 2023 at 2:33 AM Justin Sanderson via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org&gt; wrote:

>  Found the culprit.... /var/lib/ipa/ra-agent.pem
>
>
>  # openssl -in /var/lib/ipa/ra-agent.pem -noout -text |grep "Not After"
>
>  The cert expired 4 days ago. ... whats proper "IPA" way to recreate
>  cert. I could do it with openssl but idd if there's "hooks" to other
>  components that i need to update.
>
>
>  On 5/7/2023 10:08 AM, Rob Crittenden wrote:
>  > Justin Sanderson via FreeIPA-users wrote:
>  >> Ok. So once again my IPA server is having cert issues. Everything seems
>  >> to be working except when I am in the web interface and goto
>  >> "Authentication" --> "Certificates" --> Click any of
> the certs in the
>  list.
>  >>
>  >>
>  >> ---- I get this error from the browser.------
>  >>
>  >> IPA ERROR 907: NetworkError
>  >>
>  >> cannot connect to
>  >> https://[myservernamehere.fqdn]:443/ca/agent/ca/displayBySerial' :
>  >> SSL_HANDSHAKE_FAILURE
>  >>
>  >>
>  >> # getcert list |grep expires  --> everything checks out ok. no expiry on
>  >> any of the certs
>  >>
>  >>
>  >> --- checked all the certs on there "Not Before" and "Not
> After" dates
>  >> for the following NSS db's
>  >>
>  >> certutil -L -d /etc/pki/pki-tomcat/alias
>  >>
>  >> certutil -L -d /etc/httpd/alias
>  >>
>  >>
>  >>
>  >>   ---- In /var/log/httpd/error_log, I do see some errors: ----
>  >>
>  >> Bad Remote Server Certificate -8181
>  >>
>  >> SSL Library Error: -8181 Certificate has expired
>  >>
>  >>
>  >> I know it's an expired cert obviously from httpd errorlog but where is
>  >> the darn thing. I thought i checked all the places and looked ok but I'm
>  >> definitely missing something....
>  >>
>  >>
>  >> could use some advice.
>  > I'd simplify by trying on the command line: ipa cert-show 1
>  >
>  > This will exercise the basic connectivity and will be less noisy than
>  > using the UI. I'd run the same command on all servers you have in case
>  > only one is affected.
>  >
>  > As for the TLS error in the httpd.log its hard to say without broader
>  > context. Is there an access log entry at the same time which may
>  correlate?
>  >
>  > rob
>  >
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>

[Justin Sanderson]

Hey Flo - thanks so much for your willingness to help.


My setup is just a single VM server. I will give it a try tonight once 
everyone has gone home for the day.

Also, does it make sense to have certmonger monitor this cert? I found  
a command on the RH access portal that shows how to add it to certmonger 
but I had doubts about whether it would update LDAP when the cert got 
renewed...


Thanks again for the help and i'll report back the result tonight.

[Florence Blanc-Renaud]

Hi,

On Tue, May 9, 2023 at 1:24 PM Justin Sanderson via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org&gt; wrote:

>  Hey Flo - thanks so much for your willingness to help.
>
>
>  My setup is just a single VM server. I will give it a try tonight once
>  everyone has gone home for the day.
>
>  Also, does it make sense to have certmonger monitor this cert? I found
>  a command on the RH access portal that shows how to add it to certmonger
>  but I had doubts about whether it would update LDAP when the cert got
>  renewed...
>
>  By default the cert should already be tracked by certmonger. If you run *getcert
list*,you should see it in the list of tracked certs. For instance on my
system:
# getcert list -f /var/lib/ipa/ra-agent.pem
Number of certificates and requests being tracked: 12.
Request ID '20230324140132':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=IPA.TEST
subject: CN=IPA RA,O=IPA.TEST
issued: 2022-05-31 12:32:55 UTC
expires: 2024-05-20 12:32:55 UTC
key usage: digitalSignature,keyEncipherment,dataEncipherment
eku: id-kp-clientAuth
profile: caSubsystemCert
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
With a single server (that is the renewal master), certmonger renewal
should put the new cert directly in the file /var/lib/ipa/ra-agent.pem (and
then later in LDAP but that's not relevant in this case).
flo

>  Thanks again for the help and i'll report back the result tonight.
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>