12:24 a.m.
Went to renew an externally-signed IPA CA certificate that was valid
through today, and discovered that FreeIPA had decided to renew it with a
self-signed cert a month ago, and had since reissued all other subsystem
certs against that self-signed CA. After running through the
ipa-cacert-manage renew dance and ipa-certupdate, the system store now
contains the following certs, in this order:
- old, now-expired IPA CA cert
- old, soon-to-be-expired external CA root cert
- self-signed IPA cert
- new IPA CA cert
- new external CA root cert
There's also a chicken-and-egg problem with trying to renew anything, in
that all new requests are signed with the self-signed IPA CA instead of
the new intermediate IPA CA.
How do I unravel this, and completely purge the self-signed cert from
existence? Why did FreeIPA try to renew the intermediate CA cert on its
own, and why did it succeed?
(This is FreeIPA 4.7.2 on Fedora 29, which I'm stuck with until the CA
chains are sorted out -- upgrading is still a manual replica replacement
process, since ipa-server-upgrade and friends *still* insist on verifying
a CA lifetime of >2 years, inexplicable behavior reported years ago...)
-Rob
7:36 a.m.
On 1/2/20 7:24 AM, Rob Foehl via FreeIPA-users wrote:
Went to renew an externally-signed IPA CA certificate that was valid
through today, and discovered that FreeIPA had decided to renew it with
a self-signed cert a month ago, and had since reissued all other
subsystem certs against that self-signed CA.
That is surprising, maybe there was a
tracking request that triggered
the renewal to self-signed. Can you check now if the self-signed CA is
tracked? (It should not)
After running through the
ipa-cacert-manage renew dance and ipa-certupdate, the system store now
contains the following certs, in this order:
- old, now-expired IPA CA cert
- old, soon-to-be-expired external CA root cert
- self-signed IPA cert
- new IPA CA cert
- new external CA root cert
There's also a chicken-and-egg problem with trying to renew anything, in
that all new requests are signed with the self-signed IPA CA instead of
the new intermediate IPA CA.
As far as I understand, the private key of IPA CA does
not change even
when the CA is renewed from self-signed to externally-signed (or the
reverse), and this means that the same key is used to issue the IPA
certs. In that case, there is no difference if a cert was signed with
the old or the new CA cert.
What makes you think that the new requests are signed with the
self-signed IPA CA?
Do you have any issue when you try to renew other certs?
flo
How do I unravel this, and completely purge the self-signed cert from
existence? Why did FreeIPA try to renew the intermediate CA cert on its
own, and why did it succeed?
(This is FreeIPA 4.7.2 on Fedora 29, which I'm stuck with until the CA
chains are sorted out -- upgrading is still a manual replica replacement
process, since ipa-server-upgrade and friends *still* insist on
verifying a CA lifetime of >2 years, inexplicable behavior reported
years ago...)
-Rob
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
11:47 a.m.
On Thu, 2 Jan 2020, Florence Blanc-Renaud wrote:
On 1/2/20 7:24 AM, Rob Foehl via FreeIPA-users wrote:
> Went to renew an externally-signed IPA CA certificate that was valid
> through today, and discovered that FreeIPA had decided to renew it with a
> self-signed cert a month ago, and had since reissued all other subsystem
> certs against that self-signed CA.
That is surprising, maybe there was a tracking request that triggered
the
renewal to self-signed. Can you check now if the self-signed CA is tracked?
(It should not)
There's one like this, which presumably was the case a month ago when the
self-signed CA was generated:
Request ID '20190325054235':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: <external CA>
subject: <IPA CA>
expires: 2022-12-31 17:47:12 EST
key usage: keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca"
track: yes
auto-renew: yes
This system has never (intentionally) used a self-signed CA, initial
install was with the older external CA. This system was (re)installed as
a replica about a year ago, and last upgrade attempt was on 20190325.
> There's also a chicken-and-egg problem with trying to renew
anything, in
> that all new requests are signed with the self-signed IPA CA instead of
> the new intermediate IPA CA.
As far as I understand, the private key of IPA CA does not change
even when
the CA is renewed from self-signed to externally-signed (or the reverse), and
this means that the same key is used to issue the IPA certs. In that case,
there is no difference if a cert was signed with the old or the new CA cert.
The unchanging private key is its own issue, but this is the reason why I
didn't notice this a month earlier and haven't had more trouble since.
What makes you think that the new requests are signed with the
self-signed
IPA CA?
Certificate lifetimes aren't bound to the new intermediate IPA CA.
Do you have any issue when you try to renew other certs?
I can't renew any of them against the proper CA, nor are correct chains
returned to clients. The question remains: how do I get rid of the
self-signed CA entirely?
-Rob
3:58 a.m.
New subject: External CA renewal and self-signed surprise
On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
The question remains: how do I get rid of the self-signed CA
entirely?
Best hint toward this I've managed to find thus far is in the comments on
https://pagure.io/freeipa/issue/7283 , with got me as far as the
cACertificate and ipaCertIssuerSerial entries corresponding to the
extraneous self-signed cert... If I remove those and the cert from the
NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
What do I do about the rest of the mess it's created, and/or preventing
future problems? Bug ticket for the erroneous self-signed renewal?
-Rob
8:33 a.m.
On 1/13/20 10:58 AM, Rob Foehl via FreeIPA-users wrote:
On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
> The question remains: how do I get rid of the self-signed CA entirely?
Hi Rob,
there is currently no easy way to do this, except using a lot of manual
commands. For info, we already have a ticket to enhance
ipa-cacert-manage with a subcommand allowing to remove a cert:
https://pagure.io/freeipa/issue/8124
Best hint toward this I've managed to find thus far is in the
comments
on https://pagure.io/freeipa/issue/7283 , with got me as far as the
cACertificate and ipaCertIssuerSerial entries corresponding to the
extraneous self-signed cert... If I remove those and the cert from the
NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
What do I do about the rest of the mess it's created, and/or preventing
future problems? Bug ticket for the erroneous self-signed renewal?
If the issue is reproducible, then yes, we do need to fix the problem.
Please open an issue at https://pagure.io/freeipa/new_issue
Thanks,
flo
-Rob
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
9:43 p.m.
New subject: External CA renewal and self-signed surprise
On Thu, 16 Jan 2020, Florence Blanc-Renaud wrote:
On 1/13/20 10:58 AM, Rob Foehl via FreeIPA-users wrote:
> On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
>
>> The question remains: how do I get rid of the self-signed CA entirely?
>
Hi Rob,
there is currently no easy way to do this, except using a lot of manual
commands. For info, we already have a ticket to enhance ipa-cacert-manage
with a subcommand allowing to remove a cert:
https://pagure.io/freeipa/issue/8124
Okay, so... What's the process? What are the commands?
Is it going to be less pain than ripping out the entire IPA environment
and starting over, only to run into this same mess in another two years?
I agree that there ought to be an easy way (e.g. ipa-cacert-manage delete)
to do this, but at the moment, I need *any* way to do this -- been
fighting with a crippled system for three weeks now.
> Best hint toward this I've managed to find thus far is in
the comments on
> https://pagure.io/freeipa/issue/7283 , with got me as far as the
> cACertificate and ipaCertIssuerSerial entries corresponding to the
> extraneous self-signed cert... If I remove those and the cert from the
> NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
>
> What do I do about the rest of the mess it's created, and/or preventing
> future problems? Bug ticket for the erroneous self-signed renewal?
>
If the issue is reproducible, then yes, we do need to fix the problem. Please
open an issue at https://pagure.io/freeipa/new_issue
It is, as confirmed by installing a new F31 test VM and going through the
external CA ipa-server-install. https://pagure.io/freeipa/issue/8176
(In the process, I also discovered that the install fails with
CA_UNREACHABLE errors unless the system's hostname is actually resolvable
in the DNS -- despite ipa-server-install still insisting on mangling
/etc/hosts unnecessarily, as reported in
https://pagure.io/freeipa/issue/6984 years ago. Come on.)
-Rob
5:17 p.m.
On Mon, Jan 13, 2020 at 04:58:05AM -0500, Rob Foehl via FreeIPA-users wrote:
On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
> The question remains: how do I get rid of the self-signed CA entirely?
Best hint toward this I've managed to find thus far is in the comments on
https://pagure.io/freeipa/issue/7283 , with got me as far as the
cACertificate and ipaCertIssuerSerial entries corresponding to the
extraneous self-signed cert... If I remove those and the cert from the
NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
If the IPA CA's key and subject did not change, then there is no
need to reissue end-entity or other subordinate certificates. Only
the IPA CA certificate needs to be renewed (from self-signed to
externally signed) and distributed.
Cheers,
Fraser
6:54 p.m.
New subject: External CA renewal and self-signed surprise
On Mon, 20 Jan 2020, Fraser Tweedale wrote:
On Mon, Jan 13, 2020 at 04:58:05AM -0500, Rob Foehl via FreeIPA-users
wrote:
> On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
>
>> The question remains: how do I get rid of the self-signed CA entirely?
>
> Best hint toward this I've managed to find thus far is in the comments on
> https://pagure.io/freeipa/issue/7283 , with got me as far as the
> cACertificate and ipaCertIssuerSerial entries corresponding to the
> extraneous self-signed cert... If I remove those and the cert from the
> NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
>
If the IPA CA's key and subject did not change, then there is no
need to reissue end-entity or other subordinate certificates. Only
the IPA CA certificate needs to be renewed (from self-signed to
externally signed) and distributed.
I did that already. Newly (re)issued certificates do not have their
expiration times bound to the externally-signed CA. Anything with copies
of both CA certs (as fetched by ipa-certupdate, which in and of itself is
a nightmare) feeds only the self-signed CA chain to clients, not the
correct intermediate cert, breaking everything that only knows about the
external root.
Any chance we could just stick to the question of how to completely purge
the self-signed cert from existence?
-Rob
8:24 a.m.
On 1/20/20 1:54 AM, Rob Foehl via FreeIPA-users wrote:
On Mon, 20 Jan 2020, Fraser Tweedale wrote:
> On Mon, Jan 13, 2020 at 04:58:05AM -0500, Rob Foehl via FreeIPA-users
> wrote:
>> On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
>>
>>> The question remains: how do I get rid of the self-signed CA entirely?
>>
>> Best hint toward this I've managed to find thus far is in the
>> comments on
>> https://pagure.io/freeipa/issue/7283 , with got me as far as the
>> cACertificate and ipaCertIssuerSerial entries corresponding to the
>> extraneous self-signed cert... If I remove those and the cert from the
>> NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
>>
> If the IPA CA's key and subject did not change, then there is no
> need to reissue end-entity or other subordinate certificates. Only
> the IPA CA certificate needs to be renewed (from self-signed to
> externally signed) and distributed.
I did that already. Newly (re)issued certificates do not have their
expiration times bound to the externally-signed CA. Anything with
copies of both CA certs (as fetched by ipa-certupdate, which in and of
itself is a nightmare) feeds only the self-signed CA chain to clients,
not the correct intermediate cert, breaking everything that only knows
about the external root.
Any chance we could just stick to the question of how to completely
purge the self-signed cert from existence?
Sure, you can follow a manual process to remove the self-signed cert:
1- use ldapmodify in order to remove the cert from the LDAP database.
You need first to find the exact dn, and then the exact
cACertificate;binary attribute to delete. It will be stored below
cn=certificates,cn=ipa,cn=etc,$BASEDN.
2- on all the IPA servers, use "certutil -D -d </path/to/db> -n
<nickname>" to remove the cert from the following databases:
/etc/dirsrv/slapd-DOMAIN-COM
/etc/httpd/alias
/etc/pki/pki-tomcat/alias/
/etc/ipa/nssdb
3- on all IPA servers and clients, run ipa-certupdate, this command will
remove the cert from
/usr/share/ipa/html/ca.crt
/var/kerberos/krb5kdc/cacert.pem
/etc/ipa/ca.crt
/var/lib/ipa-client/pki/kdc-ca-bundle.pem
/var/lib/ipa-client/pki/ca-bundle.pem
But as Fraser pointed out, there is no need to re-issue the other certs.
flo
-Rob
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
1:03 p.m.
Florence Blanc-Renaud via FreeIPA-users wrote:
On 1/20/20 1:54 AM, Rob Foehl via FreeIPA-users wrote:
> On Mon, 20 Jan 2020, Fraser Tweedale wrote:
>
>> On Mon, Jan 13, 2020 at 04:58:05AM -0500, Rob Foehl via FreeIPA-users
>> wrote:
>>> On Thu, 2 Jan 2020, Rob Foehl via FreeIPA-users wrote:
>>>
>>>> The question remains: how do I get rid of the self-signed CA entirely?
>>>
>>> Best hint toward this I've managed to find thus far is in the
>>> comments on
>>> https://pagure.io/freeipa/issue/7283 , with got me as far as the
>>> cACertificate and ipaCertIssuerSerial entries corresponding to the
>>> extraneous self-signed cert... If I remove those and the cert from the
>>> NSSDBs, then what? Reissue all dependent certs in the IPA CA chain?
>>>
>> If the IPA CA's key and subject did not change, then there is no
>> need to reissue end-entity or other subordinate certificates. Only
>> the IPA CA certificate needs to be renewed (from self-signed to
>> externally signed) and distributed.
>
> I did that already. Newly (re)issued certificates do not have their
> expiration times bound to the externally-signed CA. Anything with
> copies of both CA certs (as fetched by ipa-certupdate, which in and of
> itself is a nightmare) feeds only the self-signed CA chain to clients,
> not the correct intermediate cert, breaking everything that only knows
> about the external root.
>
> Any chance we could just stick to the question of how to completely
> purge the self-signed cert from existence?
>
Sure, you can follow a manual process to remove the self-signed cert:
1- use ldapmodify in order to remove the cert from the LDAP database.
You need first to find the exact dn, and then the exact
cACertificate;binary attribute to delete. It will be stored below
cn=certificates,cn=ipa,cn=etc,$BASEDN.
2- on all the IPA servers, use "certutil -D -d </path/to/db> -n
<nickname>" to remove the cert from the following databases:
/etc/dirsrv/slapd-DOMAIN-COM
/etc/httpd/alias
/etc/pki/pki-tomcat/alias/
/etc/ipa/nssdb
3- on all IPA servers and clients, run ipa-certupdate, this command will
remove the cert from
/usr/share/ipa/html/ca.crt
/var/kerberos/krb5kdc/cacert.pem
/etc/ipa/ca.crt
/var/lib/ipa-client/pki/kdc-ca-bundle.pem
/var/lib/ipa-client/pki/ca-bundle.pem
But as Fraser pointed out, there is no need to re-issue the other certs.
He wants to re-issue them because while they are validly signed by the
right key they aren't bound by the dates of the CA apparently.
I suppose for each one you could resubmit the certmonger request to
force a new cert to be issued. The re-issued cert should conform to the
CA validity period.
rob
3:44 a.m.
New subject: External CA renewal and self-signed surprise
On Mon, 20 Jan 2020, Rob Crittenden wrote:
Florence Blanc-Renaud via FreeIPA-users wrote:
> Sure, you can follow a manual process to remove the self-signed cert:
> 1- use ldapmodify in order to remove the cert from the LDAP database.
> You need first to find the exact dn, and then the exact
> cACertificate;binary attribute to delete. It will be stored below
> cn=certificates,cn=ipa,cn=etc,$BASEDN.
>
> 2- on all the IPA servers, use "certutil -D -d </path/to/db> -n
> <nickname>" to remove the cert from the following databases:
> /etc/dirsrv/slapd-DOMAIN-COM
> /etc/httpd/alias
> /etc/pki/pki-tomcat/alias/
> /etc/ipa/nssdb
>
> 3- on all IPA servers and clients, run ipa-certupdate, this command will
> remove the cert from
> /usr/share/ipa/html/ca.crt
> /var/kerberos/krb5kdc/cacert.pem
> /etc/ipa/ca.crt
> /var/lib/ipa-client/pki/kdc-ca-bundle.pem
> /var/lib/ipa-client/pki/ca-bundle.pem
Thanks for this.
> But as Fraser pointed out, there is no need to re-issue the other
certs.
He wants to re-issue them because while they are validly signed by the
right key they aren't bound by the dates of the CA apparently.
Correct -- the lifetimes are all hosed, and the installed/served
certificate chains are wrong in numerous places. If nothing else, the
reissued certs at least make it clear which is which. Ongoing issues:
- All of the certificates with the excessive lifetimes should probably
also be removed or revoked after reissue, and I'd prefer to lose the
pile of expired certs as well
- ipa-certupdate won't work on any client which still only knows about the
now-expired IPA CA, which is more than half -- and the rest have all
three chains. There are a few CentOS 6 boxes in the mix, so I guess I
have to figure this out manually either way.
- ipa-certupdate only works at all when run as root with a valid Kerberos
ticket for an admin user, which is difficult when the root account isn't
allowed to login and/or spawn interactive shells
- All now-destroyed replicas need to be rebuilt/replaced, using that to
move up to a current 4.8.x release, and then deal with all the ID /
serial / etc. skew even though replicas are replaced with themselves
I'm still not convinced I wouldn't be better off just trashing what's left
of it and starting over, especially considering that there's only a
handful of users and that I'm going to need to break in to or replace
nearly every host anyway...
-Rob
1556
days inactive
1576
days old
freeipa-users@lists.fedorahosted.org
10 comments
4 participants
participants (4)
-
Florence Blanc-Renaud -
Fraser Tweedale -
Rob Crittenden -
Rob Foehl