[Freeipa-users] Re: 502 Server Error: Proxy Error when creating CA replica on RockyLinux 8.9

[Satish Patel]

Hi Florence,

Thanks for the reply, the following is my output of all the commands you
provided. look like something is wrong on custodia self-check command.

[root@ldap-vx-010101-4 ~]# ipa config-show | grep CA
  IPA CA servers: ldap-vx-010101-1.site5.example.com,
ldap-vx-010101-4.site5.example.com
  IPA CA renewal master: ldap-vx-010101-1.site5.example.com

[root@ldap-vx-010101-4 ~]# /usr/libexec/ipa/ipa-custodia-check  `hostname`
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Platform:
Linux-3.10.0-514.el7.x86_64-x86_64-with-centos-7.3.1611-Core
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: IPA version: 4.6.5
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: IPA vendor version:
4.6.5-11.el7.centos
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Realm: example.COM
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Host:
ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Remote server:
ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:31 ipa-custodia-tester] <WARNING>: Performing self-test
only.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: File
'/etc/ipa/default.conf' exists.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: File '/etc/krb5.keytab'
exists.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: File
'/etc/ipa/custodia/custodia.conf' exists.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: File
'/etc/ipa/custodia/server.keys' exists.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Custodia client created.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Loaded key for usage
'sig' from '/etc/ipa/custodia/server.keys'.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: JWK KID matches host's
service principal name 'host/[e-mail]
'.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Checked host LDAP keys
'host/[e-mail]' for usage sig.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Local key for usage 'sig'
matches key in LDAP.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Checked server LDAP keys
'host/[e-mail]' for usage sig.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Loaded key for usage
'enc' from '/etc/ipa/custodia/server.keys'.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: JWK KID matches host's
service principal name 'host/[e-mail]
'.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Checked host LDAP keys
'host/[e-mail]' for usage enc.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Local key for usage 'enc'
matches key in LDAP.
[2024-05-16T11:42:31 ipa-custodia-tester] <INFO>: Checked server LDAP keys
'host/[e-mail]' for usage enc.
[2024-05-16T11:42:31 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:31 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'dm/DMHash': 502 Server Error: Proxy Error.
[2024-05-16T11:42:31 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:31 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'ra/ipaCert': 502 Server Error: Proxy Error.
[2024-05-16T11:42:31 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:33 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'ca/auditSigningCert cert-pki-ca': 502 Server Error: Proxy Error.
[2024-05-16T11:42:33 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:34 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'ca/caSigningCert cert-pki-ca': 502 Server Error: Proxy Error.
[2024-05-16T11:42:34 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:36 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'ca/ocspSigningCert cert-pki-ca': 502 Server Error: Proxy Error.
[2024-05-16T11:42:36 requests.packages.urllib3.connectionpool] <INFO>:
Starting new HTTPS connection (1): ldap-vx-010101-4.site5.example.com
[2024-05-16T11:42:37 ipa-custodia-tester] <ERROR>: Failed to retrieve key
'ca/subsystemCert cert-pki-ca': 502 Server Error: Proxy Error.
[ERROR] One or more tests have failed.

# Custodia stuff is redirected to the custodia daemon
# after authentication
<Location "/ipa/keys/">
    ProxyPass "unix:/run/httpd/ipa-custodia.sock|http://localhost/keys/"
    RequestHeader set GSS_NAME %{GSS_NAME}s
    RequestHeader set REMOTE_USER %{REMOTE_USER}s
</Location>

[root@ldap-vx-010101-4 ~]# systemctl status ipa-custodia
● ipa-custodia.service - IPA Custodia Service
   Loaded: loaded (/usr/lib/systemd/system/ipa-custodia.service; disabled;
vendor preset: disabled)
   Active: active (running) since Fri 2024-05-10 20:13:53 UTC; 5 days ago
 Main PID: 16656 (ipa-custodia)
   CGroup: /system.slice/ipa-custodia.service
           └─16656 /usr/bin/python2 /usr/libexec/ipa/ipa-custodia
/etc/ipa/custodia/custodia.conf

Warning: Journal has been rotated since unit was started. Log output is
incomplete or unavailable.

On Thu, May 16, 2024 at 2:05 AM Florence Blanc-Renaud 
wrote:
> Hi,
>
> On Thu, May 16, 2024 at 4:42 AM Satish Patel via FreeIPA-users <
> [e-mail]> wrote:
> > Folks,
> >
> > I have Master freeIPA running on CentOS 7 and now trying to migrate it to
> > RockyLinux 8.9 (because centos7 is EOL).
> >
> > When I am running  # ipa-replica-install --setup-ca I encounter following
> > error
> >
> > Custodia uses 'ldap-vx-010101-4.site5.example.com' as master peer.
>
> Is the above node running the CA instance? You can check with
> # ipa config-show | grep CA
>   IPA CA servers: server.ipa.test
>   IPA CA renewal master: server.ipa.test
>
> Then on this "master peer" machine, check that the custodia service is
> able to find all the keys:
> # /usr/libexec/ipa/ipa-custodia-check  `hostname`
>
> I would also check the redirection for ipa/keys that should be defined in
> /etc/httpd/conf.d/ipa.conf. You should see lines similar to the following
> on the "master peer":
> # Custodia stuff is redirected to the custodia daemon
> # after authentication
> <Location "/ipa/keys/">
>     ProxyPass "unix:/run/httpd/ipa-custodia.sock|http://localhost/keys/"
>     RequestHeader set GSS_NAME %{GSS_NAME}s
>     RequestHeader set REMOTE_USER %{REMOTE_USER}s
>
> </Location>
>
> And check that the custodia service is running on this "master peer":
> # systemctl status ipa-custodia
>
> flo
>
> Configuring ipa-custodia
> > [1/4]: Generating ipa-custodia config file
> >   [2/4]: Generating ipa-custodia keys
> >   [3/4]: starting ipa-custodia
> >   [4/4]: configuring ipa-custodia to start on boot
> > Done configuring ipa-custodia.
> > Your system may be partly configured.
> > Run /usr/sbin/ipa-server-install --uninstall to clean up.
> >
> > 502 Server Error: Proxy Error for url:
> > https://ldap-vx-010101-4.site5.example.com/ipa/keys/ca/caSigningCert%20cert-...
> >
> > I did google and found a similar issue but no solutions. Any idea what
> > could be wrong here? I have checked and all certs are updated and not
> > expired.
> >
> > Above error isn't great to understand what is going on. I am able to use
> > curls etc. That means cert is updated and valid.
> > --
> > _______________________________________________
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to
> > [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > Do not reply to spam, report it:
> > https://pagure.io/fedora-infrastructure/new_issue