Hi!
Yes, seems like there was "security: off" but that doesn't seem to do it, I
think I have ended up in the situation that I need to recreate some certificates,
because:
I check the renewal dates.
--
getcert list |grep expires:
expires: 2018-03-21 09:42:04 UTC
expires: 2036-03-31 08:42:02 UTC
expires: 2018-03-21 09:42:29 UTC
expires: 2018-06-27 07:01:38 UTC
expires: 2020-08-17 10:17:32 UTC
expires: 2020-06-28 05:49:50 UTC
--
I timejump to "before oldest expired" = 2018-03-20. Dirsrv seems to start ok.
Certmonger restarts ok.
Httpd does not start. Error from /etc/httpd/logs/error_log:
--
[Tue Mar 20 07:44:39.500363 2018] [:warn] [pid 11961] NSSSessionCacheTimeout is
deprecated. Ignoring.
[Tue Mar 20 07:44:39.688595 2018] [:error] [pid 11961] SSL Library Error: -8181
Certificate has expired
[Tue Mar 20 07:44:39.688637 2018] [:error] [pid 11961] Unable to verify certificate
'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server
can start until the problem can be resolved.
--
Seems like httpd has managed to renew some certificates at some point:
--
certutil -L -d /etc/httpd/alias/ -n Server-Cert |grep Not
Not Before: Thu Jun 28 05:49:50 2018
Not After : Sun Jun 28 05:49:50 2020
--
Should I remove httpd certificate to be able to start httpd in "before 21-03-2018? I
can't seem to be able to renew these 2 (ipaCert, ocspSigningCert) without httpd
because if I try to resubmit them I get "CA_Unreachable"?
You can temporarily allow httpd to start even with expired/not yet valid
certificates: edit /etc/httpd/conf.d/nss.conf and set the
NSSEnforceValidCerts parameter to off, then restart httpd. (Do not
forget to revert the setting when you have fixed everything). See [1]
for more information.
HTH,
flo
[1]
Eemeli
-----Original Message-----
From: Florence Blanc-Renaud [mailto:flo@redhat.com]
Sent: torstai 16. elokuuta 2018 21.54
To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>; Rob Crittenden
<rcritten(a)redhat.com>
Cc: Jokinen Eemeli <Eemeli.Jokinen(a)cinia.fi>
Subject: Re: [Freeipa-users] Re: Problems after IPA upgrade: ipa-server-upgrade
doesn't complete, pki-tomcatd won't start
On 08/15/2018 01:20 PM, Jokinen Eemeli via FreeIPA-users wrote:
> Hi!
>
> Anybody can help me with this one?
>
> Summary:
>
> 2 node freeipa server cluster, node 2 was initially down for other reasons and node
1 (renewal master) had forgot to update own certificates which resulted faulty cluster.
With help from mailing list we got the node 1 back online and it's working great! Now
I'm trying to get node2 back to working order in cluster but it won't update the
certificates even when trying the timejump. Seems like it tries to renew certificates
locally although somehow I tought that it should renew the certificates from node 1...?
>
Hi,
you probably have a combination of multiple issues on your second node.
The ipa-server-upgrade failure may leave your instance in a wrong state, where dse.ldif
has disabled the ports for 389-ds (see BZ
https://bugzilla.redhat.com/show_bug.cgi?id=1569011 or pagure ticket
https://pagure.io/freeipa/issue/7534).
During the upgrade, dse.ldif is edited in order to temporarily disable the LDAP ports (to
prevent ldap modifications during the upgrade).
Sometimes, if the upgrade fails, dse.ldif is not restored and the ports remain disabled.
You will have to stop the ldap server, manually edit dse.ldif (in
/etc/dirsrv/slapd-DOMxxx) and set:
nsslapd-port: 389
nsslapd-security: on
then restart the LDAP server.
For the cert renewal, your procedure is the valid one. The kerberos error is probably
linked to 389-ds not being accessible.
HTH,
flo
>
> Eemeli
>
> -----Original Message-----
> From: Jokinen Eemeli
> Sent: keskiviikko 4. heinäkuuta 2018 16.08
> To: 'Rob Crittenden' <rcritten(a)redhat.com>; FreeIPA users list
> <freeipa-users(a)lists.fedorahosted.org>; Florence Blanc-Renaud
> <flo(a)redhat.com>
> Subject: RE: [Freeipa-users] Re: Problems after IPA upgrade:
> ipa-server-upgrade doesn't complete, pki-tomcatd won't start
>
> Hi!
>
> I reply to this since there's some data in this message queue already related to
my problem:
>
> I had 2 ipa node cluster, where the second node had been offline for some time and at
some point we had an error while trying to reboot node1 which was a Renewal Master. The
issue was that some certs had expired and after a bit of special work we got the node1
back on track. I can spot three problems and I can't (again) figure out which one is
the cause and which one I should repair first.
>
> Now I got assigned the case to get the node2 back on track also. It had some
certificates expired (obviously) so I did a small time jump and some of the certs were
renewed. However not all of them were upgraded. "getcert list" reports 3 certs
"CA Unreachable", other 3 certs seem fine.
>
> --
> getcert list |grep -A 10 "CA_UNREACH"
> status: CA_UNREACHABLE
> ca-error: Internal error
> stuck: no
> key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
> certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=Certificate Authority,O=<<REALM>>
> subject: CN=OCSP Subsystem,O=<<REALM>>
> expires: 2018-03-21 09:42:04 UTC
> key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
> eku: id-kp-OCSPSigning
> --
> status: CA_UNREACHABLE
> ca-error: Internal error
> stuck: no
> key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=Certificate Authority,O=<<REALM>>
> subject: CN=IPA RA,O=<<REALM>>
> expires: 2018-03-21 09:42:29 UTC
> key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> eku: id-kp-serverAuth,id-kp-clientAuth
> --
> status: CA_UNREACHABLE
> ca-error: Error 7 connecting to
http://<<ipa2.fqdn>>:8080/ca/ee/ca/profileSubmit: Couldn't connect to
server.
> stuck: no
> key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
> certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
> CA: dogtag-ipa-renew-agent
> issuer: CN=Certificate Authority,O=<<REALM>>
> subject: CN=<<ipa2.fqdn>>,O=<<REALM>>
> expires: 2018-06-27 07:01:38 UTC
> key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
> --
>
> Seems like "Server-Cert cert-pki-ca" is trying to renew on itself (node2)
but shouldn't node1 be the renewal master? Restarting httpd, certmonger and pki-tomcat
don't seem to help, time traveling helped on other certs but not on these.
>
> Directory service seems to work if I start it manually but ipa-server-upgrade fails
on directory server not starting with "No ports specified" so something wrong
with it or is it the certificates?
> --
> ipa-server-upgrade
> Upgrading IPA:. Estimated time: 1 minute 30 seconds
> [1/10]: stopping directory server
> [2/10]: saving configuration
> [3/10]: disabling listeners
> [4/10]: enabling DS global lock
> [5/10]: starting directory server
>
> --
> <<ipa2.fqdn>> ns-slapd[24503]: [04/Jul/2018:13:43:48.829927675 +0300] -
EMERG - main - Fatal Error---No ports specified. Exiting now.
> --
>
> Also certmonger has issues:
> --
> dogtag-ipa-ca-renew-agent-submit[1892]: Traceback (most recent call last):
>
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 541,
in <module>
>
sys.exit(main())
>
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 515,
in main
>
kinit_keytab(principal, paths.KRB5_KEYTAB, ccache_filename)
>
File "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 43,
in kinit_keytab
>
cred = gssapi.Credentials(name=name, store=store, usage='initiate')
>
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 64, in
__new__
>
store=store)
>
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 148, in
acquire
>
usage)
>
File "ext_cred_store.pyx", line 182, in
gssapi.raw.ext_cred_store.acquire_cred_from (gssapi/raw/ext_cred_store.c:1732)
>
GSSError: Major (851968): Unspecified GSS failure. Minor code may provide more
information, Minor (2529639068): Cannot contact any KDC for realm
'<<REALM>>'
> --
>
> but KDCs should be able to be resolved even from ipa node2
> --
> nslookup -type=srv _kerberos._tcp.<<REALM>>
> Server: <<ipa1.ip>>
> Address: <<ipa1.ip>>#53
>
> _kerberos._tcp.<<REALM>> service = 0 100 88
<<ipa1.fqdn>>.
> _kerberos._tcp.<<REALM>> service = 0 100 88
<<ipa2.fqdn>>.
> --
>
> For testing purposes I turned off firewall on ipa node1
>
>
> Eemeli
>
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> List Guidelines:
>
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedo
>
rahosted.org/message/XOUL2VQ26BKQHNY2XB3CDSJRKYQCHJ3X/
>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorah...