[Freeipa-users] Re: How to add another host as CA?

[Boris]

woa. It worked. Thanks a lot!
Now I can step by step upgrade to the latest fedora.

Do I need to stop all IPA instances to reset the directory manager password?

Am Do., 20. Feb. 2025 um 18:56 Uhr schrieb Florence Blanc-Renaud <
[e-mail]>:
> Hi,
>
> I think I spotted the issue: https://pagure.io/freeipa/issue/9381
>
> It was fixed on the ipa-4-10 branch but never released in ipa 4.10. Since
> your ipa1 host has freeipa-server-4.10.1-1.fc37 it doesn't have the patch.
>
> Check if you have a drop-in file
>
> /etc/systemd/system/[e-mail].d/ipa.conf
>
> If not, create one with the following content:
>
> # cat /etc/systemd/system/[e-mail].d/ipa.conf
> [Service]
> Environment=LC_ALL=C.UTF-8
> ExecStartPost=/usr/libexec/ipa/ipa-pki-wait-running
>
> then launch systemctl --system daemon-reload and ipa-server-upgrade.
>
> HTH,
> flo
>
> On Thu, Feb 20, 2025 at 5:29 PM Boris  wrote:
> > Hey,
> >
> > *pki-server subsystem-show ca *and *curl  --cert
> > /var/lib/ipa/ra-agent.pem --key /var/lib/ipa/ra-agent.key
> > https://%60hostname%60:8443/ca/rest/account/login*
> > gave the expected results.
> >
> > My thought was that there is no ca available during the update, and thats
> > why I wanted to add the 2nd host as CA.
> >
> > I feel a bit nervous about posting both logs, because it feels hard to
> > clean them up from some information.
> > I gave my best. You can find both logs here:
> > https://blktrace.kervyn.de/debug.2025-02-20.log.gz
> > https://blktrace.kervyn.de/ipaupgrade.log.gz
> >
> > Am Do., 20. Feb. 2025 um 16:53 Uhr schrieb Florence Blanc-Renaud <
> > [e-mail]>:
> > > Hi,
> > >
> > > On Wed, Feb 19, 2025 at 11:57 AM Boris  wrote:
> > > > Hi flo,
> > > >
> > > > `ipa cert-show 1` works on both IPA hosts and returns correct data,
> > > > from what I can tell.
> > >
> > > That's strange because cert-show is also authenticating to the CA REST
> > > API.
> > > > `ipa config-show` gies the following:
> > > >   IPA masters: ipa1.redacted, ipa2.redacted
> > > >   IPA master capable of PKINIT: ipa2.redacted
> > > >   IPA CA servers: ipa1.redacted
> > > >   IPA CA renewal master: ipa1.redacted
> > > >   IPA DNS servers: ipa1.redacted, ipa2.redacted
> > > >
> > > > regarding the named crashes: I think the problem might be related to
> > > > ldap. The last time the named daemons were in a restart/crash loop I
> > > > restarted the ipa2 host which immediately resolved the problem.
> > > > ipa1:
> > > > bind-9.18.19-1.fc37.x86_64
> > > > bind-dyndb-ldap-11.10-17.fc37.x86_64
> > > >
> > > > ipa2:
> > > > bind-9.16.28-1.fc35.x86_64
> > > > bind-dyndb-ldap-11.9-12.fc35.x86_64
> > > >
> > > > for the coredump I would need your guidance what to do, because I am
> > > > not that firm with named debugging.
> > > >
> > > > Here are the last couple of line from the /var/log/ipaupgrade.log file.
> > > > The update seems to go through, but it fails when it needs to authenticate
> > > > with the CA REST API
> > > >
> > > > 2025-01-21T20:24:14Z DEBUG stderr=
> > > > 2025-01-21T20:24:14Z DEBUG Starting external process
> > > > 2025-01-21T20:24:14Z DEBUG args=['/bin/systemctl', 'start',
> > > > 'certmonger.service']
> > > > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0
> > > > 2025-01-21T20:24:15Z DEBUG stdout=
> > > > 2025-01-21T20:24:15Z DEBUG stderr=
> > > > 2025-01-21T20:24:15Z DEBUG Starting external process
> > > > 2025-01-21T20:24:15Z DEBUG args=['/bin/systemctl', 'is-active',
> > > > 'certmonger.service']
> > > > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0
> > > > 2025-01-21T20:24:15Z DEBUG stdout=active
> > > >
> > > > 2025-01-21T20:24:15Z DEBUG stderr=
> > > > 2025-01-21T20:24:15Z DEBUG Start of certmonger.service complete
> > > > 2025-01-21T20:24:15Z DEBUG Starting external process
> > > > 2025-01-21T20:24:15Z DEBUG args=['pki-server', 'subsystem-show', 'kra']
> > > > 2025-01-21T20:24:15Z DEBUG Process finished, return code=1
> > > > 2025-01-21T20:24:15Z DEBUG stdout=
> > > > 2025-01-21T20:24:15Z DEBUG stderr=ERROR: ERROR: No kra subsystem in
> > > > instance pki-tomcat.
> > > >
> > > > 2025-01-21T20:24:15Z INFO [Update certmonger certificate renewal
> > > > configuration]
> > > > 2025-01-21T20:24:15Z DEBUG Loading Index file from
> > > > '/var/lib/ipa/sysrestore/sysrestore.index'
> > > > 2025-01-21T20:24:15Z DEBUG Starting external process
> > > > 2025-01-21T20:24:15Z DEBUG args=['/usr/bin/certutil', '-d',
> > > > 'sql:/etc/dirsrv/slapd-redacted/', '-L', '-n', 'Server-Cert', '-a', '-f',
> > > > '/etc/dirsrv/slapd-redacted/pwdfile.txt']
> > > > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0
> > > > 2025-01-21T20:24:15Z DEBUG stdout=-----BEGIN CERTIFICATE-----
> > > > redacted
> > > > -----END CERTIFICATE-----
> > > >
> > > > 2025-01-21T20:24:15Z DEBUG stderr=
> > > > 2025-01-21T20:24:15Z DEBUG Loading Index file from
> > > > '/var/lib/ipa/sysrestore/sysrestore.index'
> > > > 2025-01-21T20:24:15Z DEBUG Starting external process
> > > > 2025-01-21T20:24:15Z DEBUG args=['/usr/bin/certutil', '-d',
> > > > 'sql:/etc/pki/pki-tomcat/alias', '-L', '-f',
> > > > '/etc/pki/pki-tomcat/alias/pwdfile.txt']
> > > > 2025-01-21T20:24:15Z DEBUG Process finished, return code=0
> > > > 2025-01-21T20:24:15Z DEBUG stdout=
> > > > Certificate Nickname                                         Trust
> > > > Attributes
> > > >
> > > > SSL,S/MIME,JAR/XPI
> > > >
> > > > caSigningCert cert-pki-ca                                    CTu,Cu,Cu
> > > > caSigningCert cert-pki-ca 6148bb27-6bd6-4a0a-b607-6ba538a6c401 u,u,u
> > > > ocspSigningCert cert-pki-ca                                  u,u,u
> > > > subsystemCert cert-pki-ca                                    u,u,u
> > > > auditSigningCert cert-pki-ca                                 u,u,Pu
> > > > Server-Cert cert-pki-ca                                      u,u,u
> > > >
> > > > 2025-01-21T20:24:15Z DEBUG stderr=
> > > > 2025-01-21T20:24:15Z INFO Certmonger certificate renewal configuration
> > > > already up-to-date
> > > > 2025-01-21T20:24:15Z INFO [Enable PKIX certificate path discovery and
> > > > validation]
> > > > 2025-01-21T20:24:15Z DEBUG Loading StateFile from
> > > > '/var/lib/ipa/sysupgrade/sysupgrade.state'
> > > > 2025-01-21T20:24:15Z INFO PKIX already enabled
> > > > 2025-01-21T20:24:15Z INFO [Authorizing RA Agent to modify profiles]
> > > > 2025-01-21T20:24:15Z INFO [Authorizing RA Agent to manage lightweight
> > > > CAs]
> > > > 2025-01-21T20:24:15Z INFO [Ensuring Lightweight CAs container exists in
> > > > Dogtag database]
> > > > 2025-01-21T20:24:15Z INFO [Adding default OCSP URI configuration]
> > > > 2025-01-21T20:24:15Z INFO [Disabling cert publishing]
> > > > 2025-01-21T20:24:15Z INFO [Ensuring CA is using LDAPProfileSubsystem]
> > > > 2025-01-21T20:24:15Z INFO [Migrating certificate profiles to LDAP]
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'AdminCert' is already in LDAP and
> > > > enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'DomainController' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'ECAdminCert' is already in LDAP and
> > > > enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'acmeServerCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caAdminCert' is already in LDAP and
> > > > enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caAgentFileSigning' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caAgentServerCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caAuditSigningCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCACert' is already in LDAP and
> > > > enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCECUserCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCECserverCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCECsubsystemCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCMCUserCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caCrossSignedCACert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDirBasedDualCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDirPinUserCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDirUserCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDirUserRenewal' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDualCert' is already in LDAP and
> > > > enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caDualRAuserCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caECAdminCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caECAgentServerCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caECDirPinUserCert' is already in
> > > > LDAP and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caECDirUserCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z DEBUG Profile 'caECDualCert' is already in LDAP
> > > > and enabled; skipping
> > > > 2025-01-21T20:24:15Z INFO Migrating profile 'caECFullCMCSharedTokenCert'
> > > > 2025-01-21T20:24:15Z DEBUG request GET
> > > > https://ipa1.redacted:8443/ca/rest/account/login
> > > > 2025-01-21T20:24:15Z DEBUG request body ''
> > > > 2025-01-21T20:24:16Z DEBUG response status 404
> > > > 2025-01-21T20:24:16Z DEBUG response headers Content-Type:
> > > > text/html;charset=utf-8
> > > > Content-Language: en
> > > > Content-Length: 784
> > > > Date: Tue, 21 Jan 2025 20:24:16 GMT
> > > >
> > > > 2025-01-21T20:24:16Z DEBUG response body (decoded): b'<!doctype
> > > > html><html lang="en"><head><title>HTTP Status 404 \xe2\x80\x93 Not
> > > > Found</title><style type="text/css">body
> > > > {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b
> > > > {color:white;background-color:#525D76;} h1 {font-size:22px;} h2
> > > > {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;}
> > > > .line
> > > > {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP
> > > > Status 404 \xe2\x80\x93 Not Found</h1><hr class="line" /><p><b>Type</b>
> > > > Status Report</p><p><b>Message</b> The requested resource
> > > > [&#47;ca&#47;rest&#47;account&#47;login] is not
> > > > available</p><p><b>Description</b> The origin server did not find a current
> > > > representation for the target resource or is not willing to disclose that
> > > > one exists.</p><hr class="line" /><h3>Apache
> > > > Tomcat/9.0.82</h3></body></html>'
> > > > 2025-01-21T20:24:16Z ERROR IPA server upgrade failed: Inspect
> > > > /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
> > > > 2025-01-21T20:24:16Z DEBUG   File
> > > > "/usr/lib/python3.11/site-packages/ipapython/admintool.py", line 180, in
> > > > execute
> > > >     return_value = self.run()
> > > >                    ^^^^^^^^^^
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/ipa_server_upgrade.py",
> > > > line 54, in run
> > > >     server.upgrade()
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py",
> > > > line 2061, in upgrade
> > > >     upgrade_configuration()
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py",
> > > > line 1914, in upgrade_configuration
> > > >     ca_enable_ldap_profile_subsystem(ca)
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/server/upgrade.py",
> > > > line 458, in ca_enable_ldap_profile_subsystem
> > > >     cainstance.migrate_profiles_to_ldap()
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/cainstance.py", line
> > > > 2155, in migrate_profiles_to_ldap
> > > >     _create_dogtag_profile(profile_id, profile_data, overwrite=False)
> > > >   File
> > > > "/usr/lib/python3.11/site-packages/ipaserver/install/cainstance.py", line
> > > > 2209, in _create_dogtag_profile
> > > >     with api.Backend.ra_certprofile as profile_api:
> > > >   File "/usr/lib/python3.11/site-packages/ipaserver/plugins/dogtag.py",
> > > > line 1211, in __enter__
> > > >     raise errors.RemoteRetrieveError(reason=_('Failed to authenticate
> > > > to CA REST API'))
> > > >
> > > > Can you check if the CA subsystem is enabled?
> > >
> > > # *pki-server subsystem-show ca*
> > >   Subsystem ID: ca
> > >   Instance ID: pki-tomcat
> > >   Enabled: True
> > >
> > > If yes, try to authenticate to the rest API with curl:
> > > # *curl  --cert /var/lib/ipa/ra-agent.pem --key
> > > /var/lib/ipa/ra-agent.key https://%60hostname%60:8443/ca/rest/account/login*
> > > {"id":"ipara","FullName":"ipara","Roles":["Certificate Manager
> > > Agents","Enterprise ACME Administrators","Registration Manager
> > > Agents","Security Domain Administrators"],"Attributes":{"Attribute":[]}}
> > >
> > > If the above commands are working, retry the upgrade with
> > > # *ipa-server-upgrade*
> > > and send us the full /var/log/ipaupgrade.log and
> > > /var/log/pki/pki-tomcat/ca/debug.$DATE.log.
> > >
> > > flo
> > >
> > > 2025-01-21T20:24:16Z DEBUG The ipa-server-upgrade command failed,
> > > > exception: RemoteRetrieveError: Failed to authenticate to CA REST API
> > > > 2025-01-21T20:24:16Z ERROR Unexpected error - see
> > > > /var/log/ipaupgrade.log for details:
> > > > RemoteRetrieveError: Failed to authenticate to CA REST API
> > > > 2025-01-21T20:24:16Z ERROR The ipa-server-upgrade command failed. See
> > > > /var/log/ipaupgrade.log for more information
> > > >
> > > > Am Mi., 19. Feb. 2025 um 10:56 Uhr schrieb Florence Blanc-Renaud <
> > > > [e-mail]>:
> > > > > Hi,
> > > > >
> > > > > in a previous message you mentioned that the directory manager
> > > > > password is lost. You can follow this article to reset the DM password:
> > > > > https://access.redhat.com/solutions/203473
> > > > >
> > > > > Named crashes could be related to multiple issues:
> > > > > inconsistent versions between bind and bind-dyndb-ldap. Which
> > > > >
> > > > > versions do you have?
> > > > > an insufficient number of threads
> > > > > an issue when reloading the zones
> > > > >
> > > > > If you can gather a coredump and install the debug packages it could
> > > > > help identify if you're hitting a known issue.
> > > > >
> > > > > You mentioned that ipa1 needs to be started with --force, can you tell
> > > > > which service is failing and provide the logs? There should be also more
> > > > > information in /var/log/ipaupgrade.log.
> > > > >
> > > > > In order to check the CA state, a useful command is 'ipa cert-show 1'
> > > > > as it communicates with the CA to gather the certificate details. If this
> > > > > command is failing (likely with "Failed to Authenticate to CA rest API")
> > > > > you need to understand where the config is broken.
> > > > > Start by checking which system is the CA renewal master:
> > > > > ipa config-show
> > > > >
> > > > > The CA renewal master will be your priority.
> > > > >
> > > > > flo
> > > > >
> > > > > On Wed, Feb 19, 2025 at 10:25 AM Boris via FreeIPA-users <
> > > > > [e-mail]> wrote:
> > > > > > I think the CA is working, but I don't know for sure and how to
> > > > > > verify it. At least there are no expired certs on both ipa hosts
> > > > > >
> > > > > > [root@ipa1 ~]# getcert list | grep expires
> > > > > > expires: 2025-11-29 13:19:40 CET
> > > > > > expires: 2025-04-15 16:27:34 CEST
> > > > > > expires: 2025-04-15 16:26:44 CEST
> > > > > > expires: 2025-04-15 16:27:14 CEST
> > > > > > expires: 2037-08-19 16:11:12 CEST
> > > > > > expires: 2025-04-15 16:27:54 CEST
> > > > > > expires: 2025-04-15 16:27:04 CEST
> > > > > > expires: 2040-02-12 12:46:50 CET
> > > > > > expires: 2025-05-29 16:12:51 CEST
> > > > > > expires: 2026-01-26 13:48:23 CET
> > > > > >
> > > > > > [root@ipa2 ~]# getcert list | grep expires
> > > > > > expires: 2027-02-16 10:42:29 CET
> > > > > > expires: 2027-02-16 10:42:51 CET
> > > > > > expires: 2025-04-15 16:27:04 CEST
> > > > > > expires: 2027-02-16 10:43:26 CET
> > > > > >
> > > > > > The healthcheck showed some "group is not correct" and "files are to
> > > > > > permissive" which I resolved.
> > > > > > Now I have these to checks which do not tell me anything
> > > > > >       "msg": "certmonger tracking request {key} found and is not
> > > > > > expected on an IPA master."
> > > > > >       "msg": "No KDC workers defined in {sysconfig}"
> > > > > >
> > > > > > Am Di., 18. Feb. 2025 um 15:22 Uhr schrieb Rob Crittenden via
> > > > > > FreeIPA-users :
> > > > > >
> > > > > > > Boris wrote:
> > > > > > > > Hi Rob,
> > > > > > > >
> > > > > > > > I have two hosts: ipa1 and ipa2
> > > > > > > >
> > > > > > > > ipa1:
> > > > > > > > Fedora 37
> > > > > > > > freeipa-server-4.10.1-1.fc37.x86_64
> > > > > > > > Managed suffixes: domain, ca
> > > > > > > > running with ipactl start --force because the update is not
> > > > > > > working (The
> > > > > > > > ipa-server-upgrade command failed, exception: RemoteRetrieveError:
> > > > > > > > Failed to authenticate to CA REST API).
> > > > > > > > I tried to upgrade, but the upgrade did not go through.
> > > > > > >
> > > > > > > Your existing CA is having issues. I'd start by checking that your CA
> > > > > > > certificates are still valid: getcert list | grep expires
> > > > > > >
> > > > > > > You might also try installing the freeipa-healthcheck package and
> > > > > > > running ipa-healthcheck. Expect a lot of errors since it won't be
> > > > > > > able
> > > > > > > to connect to the CA but it will also check the validity dates, etc.
> > > > > > >
> > > > > > > > ipa2:
> > > > > > > > Fedora 35
> > > > > > > > freeipa-server-4.9.11-1.fc35.x86_64
> > > > > > > > Managed suffixes: domain
> > > > > > > >
> > > > > > > > So my thought process was: if it can not authenticate against the
> > > > > > > CA
> > > > > > > > REST API, I need to add the CA capability to ipa2
> > > > > > >
> > > > > > > You need to authenticate to the CA to create a clone of it. You can't
> > > > > > > install another CA until you get your existing one working.
> > > > > > >
> > > > > > > rob
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > FreeIPA-users mailing list -- [e-mail]
> > > > > > To unsubscribe send an email to
> > > > > > [e-mail]
> > > > > > Fedora Code of Conduct:
> > > > > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > > > > List Guidelines:
> > > > > > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > > > List Archives:
> > > > > > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > > > > > Do not reply to spam, report it:
> > > > > > https://pagure.io/fedora-infrastructure/new_issue
> > > >
> > > > --
> > > > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend
> > > > im groüen Saal.
> >
> > --
> > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
> > groüen Saal.