On su, 06 helmi 2022, Brian J. Murrell wrote:
On Sun, 2022-02-06 at 12:43 +0200, Alexander Bokovoy wrote:
>
>
> # Protect /ipa and everything below it in webspace with Apache Kerberos
> auth
> <Location "/ipa">
> AuthType GSSAPI
> AuthName "Kerberos Login"
> GssapiUseSessions On
> Session On
> SessionCookieName ipa_session path=/ipa;httponly;secure;
> SessionHeader IPASESSION
> # Uncomment the following to have shorter sessions, but beware this
> may break
> # old IPA client tols that incorrectly parse cookies.
> # SessionMaxAge 1800
> GssapiSessionKey file:$GSSAPI_SESSION_KEY
On both of my servers, this is actually:
GssapiSessionKey file:/etc/httpd/alias/ipasession.key
Sure. I was showing the template that is used during installation. All
$-based variables get replaced by their proper values, depending on the
deployment environment. The paths might be different in various
distributions.
On both of my servers, this timer is disabled:
# systemctl status ipa-ccache-sweep.timer
● ipa-ccache-sweep.timer - Remove Expired Kerberos Credential Caches
Loaded: loaded (/usr/lib/systemd/system/ipa-ccache-sweep.timer; disabled; vendor
preset: disabled)
Active: inactive (dead)
Trigger: n/a
Some kind of installation bug? Should these be enabled?
I think timer should be enabled during package upgrades.
%posttrans server
# don't execute upgrade and restart of IPA when server is not installed
%{__python3} -c "import sys; from ipalib import facts; sys.exit(0 if
facts.is_ipa_configured() else 1);" > /dev/null 2>&1
if [ $? -eq 0 ]; then
[other code]
/bin/systemctl is-enabled ipa-ccache-sweep.timer >/dev/null 2>&1
if [ $? -eq 1 ]; then
/bin/systemctl enable ipa-ccache-sweep.timer>/dev/null
fi
fi
> To do so, we rely on POSIX ACLs set on /run/ipa/ccaches that
allow
> apache group to read/write to the files in /run/ipa/ccaches and force
> ownership to ipaapi:ipaapi user and group.
>
> Here is an example:
> ---------------------
> [root@dc ~]# ls -la /run/ipa/ccaches/
> total 0
> drwsrws---+ 2 ipaapi ipaapi 40 Feb 4 11:11 .
> drwx--x--x. 3 root root 100 Feb 4 11:12 ..
>
> [root@dc ~]# getfacl /run/ipa/ccaches/
> getfacl: Removing leading '/' from absolute path names
> # file: run/ipa/ccaches/
> # owner: ipaapi
> # group: ipaapi
> # flags: ss-
> user::rwx
> group::rwx
> group:apache:rwx
> mask::rwx
> other::---
Perhaps this is where the problem lies then. On my working replica, I
see the same as above but on the non-webUI-working one I have:
# getfacl /run/ipa/ccaches/
getfacl: Removing leading '/' from absolute path names
# file: run/ipa/ccaches/
# owner: ipaapi
# group: ipaapi
user::rwx
group::rwx
other::---
It's missing the group:apache:rwx, but the tmpfile that should be
creating it does seem to be present:
# grep -r ccaches /usr/lib/tmpfiles.d/
/usr/lib/tmpfiles.d/ipa.conf:d /run/ipa/ccaches 6770 ipaapi ipaapi
/usr/lib/tmpfiles.d/ipa.conf:a+ /run/ipa/ccaches - - - - g:apache:rwx
So, to correct it:
# setfacl -m g:apache:rwx /run/ipa/ccaches/
# getfacl /run/ipa/ccaches/
getfacl: Removing leading '/' from absolute path names
# file: run/ipa/ccaches/
# owner: ipaapi
# group: ipaapi
user::rwx
group::rwx
group:apache:rwx
mask::rwx
other::---
You could have just re-run
systemd-tmpfiles --create /usr/lib/tmpfiles.d/ipa.conf
This should have been run on a server restart as well.
But that doesn't seem to have helped:
# kinit admin
Password for admin(a)EXAMPLE.COM
# ipa ping
ipa: ERROR: Major (851968): Unspecified GSS failure. Minor code may provide more
information, Minor (2598845122): Credentials cache permissions incorrect (filename:
/run/ipa/ccaches/admin(a)EXAMPLE.COM-wZC7kh)
# ls -la /run/ipa/ccaches/
total 24
drwxrwx---+ 2 ipaapi ipaapi 100 Feb 6 09:47 .
drwx--x--x. 3 root root 100 Feb 5 10:08 ..
-rw-rw----. 1 apache apache 5624 Feb 6 09:47 admin(a)EXAMPLE.COM-wZC7kh
/run/ipa/ccaches/ seems to also be missing the user and group suid
bits. Fixing that seems to make ipa ping at least work now:
# ipa ping
-------------------------------------------
IPA server version 4.9.6. API version 2.245
-------------------------------------------
# ls -la /run/ipa/ccaches/
total 24
drwsrws---+ 2 ipaapi ipaapi 80 Feb 6 09:54 .
drwx--x--x. 3 root root 100 Feb 5 10:08 ..
-rw-------. 1 ipaapi ipaapi 9808 Feb 6 09:54 admin(a)EXAMPLE.COM-vxMhbf
-rw-------. 1 ipaapi ipaapi 9808 Feb 6 09:54 admin(a)EXAMPLE.COM-ycq8Ye
> If you look with klist into that ccache, you'll only see encrypted
> gssproxy object data:
>
> ------------------------
> [root@dc ~]# klist -c /run/ipa/ccaches/admin(a)IPA.TEST-GEIcn3
> Ticket cache: FILE:/run/ipa/ccaches/admin@IPA.TEST-GEIcn3
> Default principal: admin(a)IPA.TEST
>
> Valid starting Expires Service principal
> 010170 00:00:00 010170 00:00:00 Encrypted/Credentials/v1@X-GSSPROXY:
I have two caches and they are:
# klist -c /run/ipa/ccaches/admin(a)EXAMPLE.COM-RJLFGU
Ticket cache: FILE:/run/ipa/ccaches/admin@EXAMPLE.COM-RJLFGU
Default principal: admin(a)EXAMPLE.COM
Valid starting Expires Service principal
1969-12-31 19:00:00 1969-12-31 19:00:00 Encrypted/Credentials/v1@X-GSSPROXY:
# klist -c /run/ipa/ccaches/admin(a)EXAMPLE.COM-V05fsT
Ticket cache: FILE:/run/ipa/ccaches/admin@EXAMPLE.COM-V05fsT
Default principal: admin(a)EXAMPLE.COM
Valid starting Expires Service principal
1969-12-31 19:00:00 1969-12-31 19:00:00 Encrypted/Credentials/v1@X-GSSPROXY:
The valid and expiration dates on mine seem to be different, but they
still seem to be working.
WebUI is still not working though. When I try to use the WebUI from a
client which I have obtained a tgt on, the client does not even get an
HTTP principle ticket, nor does the /run/ipa/ccaches/ dir on the server
I am trying to use the webUI on have anything in /run/ipa/ccaches for
that user.
The httpd error log does log:
[Sun Feb 06 10:09:08.523430 2022] [wsgi:error] [pid 85411:tid 139661198296832] [remote
fd31:aeb1:48df:0:2374:a2d1:e1b7:89a3:36228] ipa: INFO: [jsonserver_i18n_messages] UNKNOWN:
i18n_messages(version='2.245'): SUCCESS
[Sun Feb 06 10:09:08.710420 2022] [auth_gssapi:error] [pid 85414:tid 139661028603648]
[client fd31:aeb1:48df:0:2374:a2d1:e1b7:89a3:36228] Failed to unseal session data!,
referer:
https://server.example.com/ipa/ui/
[Sun Feb 06 10:09:08.730708 2022] [auth_gssapi:error] [pid 85414:tid 139660995032832]
[client fd31:aeb1:48df:0:2374:a2d1:e1b7:89a3:36228] Failed to unseal session data!,
referer:
https://server.example.com/ipa/ui/
when that webUI login attempt fails.
This (Failed to unseal session data) means there is wrong key used by
mod_auth_gssapi to encrypt the original session and to decrypt it now. I
can only assume you have been restarting server or its components and
/etc/httpd/alias/ipasession.key got regenerated?
When doing tests with reboot/removal, it is best to clear cookies on the
client side as well.
> - /run/ipa/ccaches has wrong setup, not corresponding to what is
> shown above
It was clearly wrong, as I detailed above, (which seems to be a
tmpfiles problem perhaps? I will reboot and see what it is set up as
on reboot) but those are fixed now.
> - /run/ipa/ccaches/user@REALM-suffix file is not accesible to httpd
> processes, or damaged, making its content not decryptable by
> GSSProxy
But I am not even getting that file when I try to log into the webUI.
So things are not even making it that far.
> - ipa_session cookie generated by mod_auth_gssapi earlier and
> received
> by httpd process now is encrypted by a different mod_auth_gssapi
> session key, rendering its content damaged with regards to what
> Kerberos 5 library expects for a ccache
I'm not sure how to assess if this is the problem or not.
Well, according to your httpd logs this is the case.
> - GSSProxy session key is different to what was used to
encrypt
> ccaches
> in /run/ipa/ccaches. This happens on reboot, for example.
What is the way to determine if this is the case, or at least rectify
it so that things are back in sync?
on reboot gssproxy session key is regenerated, so all files from
/run/ipa/ccaches should be invalid. But since /run/ipa/ccaches is tmpfs,
they'll be removed automatically.
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland