New subject: KRA: problems renewing 'storageCert cert-pki-kra'

14 Nov 2021


      Hi,
I'm working to get the KRA subsystem in shape again. It has been broken
due to failed system replication (which is since fixed). There might be
lurking further problems - let's see what we can find out.
I'm working through ipa-healthcheck - currently on the CA renewal
master. The (last) error I'm having there is the following:
[
  {
    "source": "ipahealthcheck.ipa.certs",
    "check": "IPADogtagCertsMatchCheck",
    "result": "ERROR",
    "uuid": "70f767e8-19de-4426-94e0-6c7704a72895",
    "when": "20211114152924Z",
    "duration": "2.801277",
    "kw": {
      "key": "storageCert cert-pki-kra",
      "nickname": "storageCert cert-pki-kra",
      "dbdir": "/etc/pki/pki-tomcat/alias",
      "msg": "{nickname} certificate in NSS DB {dbdir} does not match entry in LDAP"
    }
  }
]
[That certificate will expire in dezember.]
And that's due to an error I made when trying to fix KRA. This is an
excerpt from "getcert list":
Request ID '20210210143948':
        status: MONITORING
        ca-error: Server at "http://freeipa1.example.org:8080/ca/ee/ca/profileSubmit" replied: Policy Set Not Found
        stuck: no
        key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB',pin s
et
        certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert cert-pki-kra',token='NSS Certificate DB'
        CA: dogtag-ipa-ca-renew-agent
        issuer: CN=Certificate Authority,O=EXAMPLE.ORG
        subject: CN=KRA Storage Certificate,O=IPA.EXAMPLE.ORG
        issued: 2020-12-31 21:08:29 CET
        expires: 2022-12-21 21:08:29 CET
        key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
        eku: id-kp-clientAuth
        profile: caStorageCert
        pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
        post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "storageCert cert-pki-kra"
        track: yes
        auto-renew: yes
That's the error - I copied the request from a temporary install but
failed to fix the subject (did not remove "IPA.")
subject: CN=KRA Storage Certificate,O=IPA.EXAMPLE.ORG
I tried to resubmit the certificate request with:
getcert resubmit -i 20210210143948 -N "CN=KRA Storage Certificate,O=EXAMPLE.ORG"
Here's the redacted log:
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_REQ_SUBJECT" to "CN=KRA Storage Certificate,O=EXAMPLE.ORG" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_OPERATION" to "SUBMIT" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_CSR" to "-----BEGIN NEW CERTIFICATE REQUEST-----
<redacted>
Nov 14 15:09:57 freeipa1 certmonger[209808]: -----END NEW CERTIFICATE REQUEST-----
Nov 14 15:09:57 freeipa1 certmonger[209808]: " for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_SPKAC" to "<redacted>" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_SPKI" to "<redacted>" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_LOCAL_CA_DIR" to "/var/lib/certmonger/local" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_KEY_TYPE" to "RSA" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_CA_NICKNAME" to "dogtag-ipa-ca-renew-agent" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_CA_PROFILE" to "caStorageCert" for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_CERTIFICATE" to "-----BEGIN CERTIFICATE-----
<redacted>
Nov 14 15:09:57 freeipa1 certmonger[209808]: -----END CERTIFICATE-----
Nov 14 15:09:57 freeipa1 certmonger[209808]: " for child.
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Redirecting stdin to /dev/null, leaving stdout and stderr open for child "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit".
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Running enrollment helper "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit".
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Certificate submission still ongoing.
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Certificate submission attempt complete.
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Child status = 2.
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Child output:
Nov 14 15:09:58 freeipa1 certmonger[209234]: "Server at "http://freeipa1.example.org:8080/ca/ee/ca/profileSubmit" replied: Policy Set Not Found
Nov 14 15:09:58 freeipa1 certmonger[209234]: "
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Server at "http://freeipa1.example.org:8080/ca/ee/ca/profileSubmit" replied: Policy Set Not Found
Nov 14 15:09:58 freeipa1 certmonger[209234]: 2021-11-14 15:09:58 [209234] Certificate not (yet?) issued.
I think the request uses the correct Profile here:
Nov 14 15:09:57 freeipa1 certmonger[209808]: 2021-11-14 15:09:57 [209808] Setting "CERTMONGER_CA_PROFILE" to "caStorageCert" for child.
The CA has the profile in the filesystem:
pki/pki-tomcat/ca/CS.cfg:1145:profile.caStorageCert.config=/var/lib/pki/pki-tomcat/ca/profiles/ca/caStorageCert.cfg
The file has not been modified and is the same as on my temporary installation.
That's from the ca error log:
2021-11-14 15:09:58 [http-nio-8080-exec-15] INFO: EnrollProfile: Creating ernrollment request 139960063
2021-11-14 15:09:58 [http-nio-8080-exec-15] SEVERE: CertProcessor: no profile policy set found
2021-11-14 15:09:58 [http-nio-8080-exec-15] SEVERE: ProfileSubmitServlet: error in processing request: Policy Set Not Found
Policy Set Not Found
        at com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:376)
        at com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:189)
        at com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:97)
        at com.netscape.cms.servlet.profile.ProfileSubmitServlet.processEnrollment(ProfileSubmitServlet.java:279)
        at com.netscape.cms.servlet.profile.ProfileSubmitServlet.process(ProfileSubmitServlet.java:132)
        at com.netscape.cms.servlet.base.CMSServlet.service(CMSServlet.java:501)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
        at jdk.internal.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:221)
        at org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:145)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:143)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at jdk.internal.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:187)
        at org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:145)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:143)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540)
        at com.netscape.cms.tomcat.ExternalAuthenticationValve.invoke(ExternalAuthenticationValve.java:83)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1722)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:829)
I found dogtag bug https://github.com/dogtagpki/pki/issues/2872 , but I
don't see how we can work around the error from the IPA side.
Any idea how I can get the certificate renewed?
Jochen
-- 
This space is intentionally left blank.