Spike,

The fact that sssd-kcm was present but didn't belong to no RPM means something went wrong somewhere/sometime. Maybe, as Alexey said, the RPM was once installed, but it was removed at some time. How did the files remain, I cannot tell. That's not normal, but neither is it normal that it belongs to no RPM. Maybe it was once installed by building from the sources without using the RPMs?

My guess is that because the package was not installed, it was not updated; but all the common libraries were. This explains the missing symbols. This situation changed when the package was installed.

But all we can do is guess, because this cannot be reproduced and we don't know the exact situation before all this happened.

HTH,
Alejandro.

On Thu, Jan 11, 2024 at 5:46 PM Spike White <spikewhitetx@gmail.com> wrote:
Alejandro,

I don't have any failing servers right in front of me now, so this is from memory.  (all reported servers fixed).

> During the upgrade KCM was not updated. How was the upgrade done? Just a simple `dnf upgrade` or something more complex?

Yes, dnf update of all outstanding RPMs.

> There is a /usr/libexec/sssd/sssd_kcm file that belongs to no RPM. Are you sure there were no errors during the update? No KCM RPM installed, even the previous version?

Prior to latest round of patching, app teams were not complaining about any sssd login issues.  After patching, they were.

> Once the RPM is manually installed, KCM successfully starts. Does `kinit` still fail at this point? If KCM fails to start, it is normal that `kinit` also fails if it is configured to use KCM.

Once sssd-kcm manually installed, kinit works.  there's a file /etc/krb5.conf.d/kcm_default_ccache with this content:

# This file should normally be installed by your distribution into a
# directory that is included from the Kerberos configuration file (/etc/krb5.conf)
# On Fedora/RHEL/CentOS, this is /etc/krb5.conf.d/
#
# To enable the KCM credential cache enable the KCM socket and the service:
#   systemctl enable sssd-kcm.socket
#   systemctl start sssd-kcm.socket
#
# To disable the KCM credential cache, comment out the following lines.

[libdefaults]
    default_ccache_name = KCM:

So with the sssd-kcm RPM missing, kinit -k fails.  I can make it succeed (even with sssd-kcm service not running) via:

# export KRB5CCNAME="FILE:/tmp/krb5cc_0"

# kinit -k


This is overriding the setting in /etc/krb5.conf.d/kcm_default_ccache  above.

BTW, /etc/krb5.conf.d/kcm_default_ccache file comes from sssd-kcm RPM so that tells me that certainly at one time (prior to patching) the sssd-kcm RPM was on the server.

Spike


On Thu, Jan 11, 2024 at 3:27 AM Alejandro Lopez <allopez@redhat.com> wrote:
Hi Spike,

During the upgrade KCM was not updated. How was the upgrade done? Just a simple `dnf upgrade` or something more complex?

There is a /usr/libexec/sssd/sssd_kcm file that belongs to no RPM. Are you sure there were no errors during the update? No KCM RPM installed, even the previous version?

Once the RPM is manually installed, KCM successfully starts. Does `kinit` still fail at this point? If KCM fails to start, it is normal that `kinit` also fails if it is configured to use KCM.

Alejandro

On Wed, Jan 10, 2024 at 11:02 PM Spike White <spikewhitetx@gmail.com> wrote:
All,

Is there a packaging problem on the latest version of RHEL8 sssd?

On several of our RHEL8 servers during the last update cycle, sssd logins start failing.  It appears to be when upgrading to version sssd-2.9.1-4.0.1.el8_9.x86_64.

Upon deep dive, it turns out the sssd-kcm RPM is missing.    

The sssd-kcm service fails to start, complaining about missing symbols in /usr/libexec/sssd_kcm file.

Sure enough, there is a problem with that file:

[root@mftplat1wplp105 ~]# rpm -qf /usr/libexec/sssd/sssd_kcm

file /usr/libexec/sssd/sssd_kcm is not owned by any package


Once the sssd-kcm RPM is installed,  then this looks normal:


[root@mftplat1wplp105 ~]# rpm -qf /usr/libexec/sssd/sssd_kcm

sssd-kcm-2.9.1-4.0.1.el8_9.x86_64


Then sssd-kcm service will restart fine.  and then sssd logins again work fine.


Also, it appears that a yum downgrade of sssd seems to work too -- presumably because this downgrade also triggers an install of sssd-kcm RPM.


I see that the sssd RPM has no direct explicit RPM dependency on sssd-kcm.  But is there some indirect or implied dependency that was missed on this latest packaging for sssd-2.9.1-4.0.1.el8_9.x86_64?


We've run sssd for years and never had significant sssd-kcm problems until last month.


BTW, another clue that it was KCM is that 'kinit -k' fails with error:


# knit -k

Cannot update default cache


But if you tell it to store creds under /tmp, it works:


# export KRB5CCNAME="FILE:/tmp/krb5cc_0"

# kinit -k


Spike


--
_______________________________________________
sssd-users mailing list -- sssd-users@lists.fedorahosted.org
To unsubscribe send an email to sssd-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
Alejandro
--
_______________________________________________
sssd-users mailing list -- sssd-users@lists.fedorahosted.org
To unsubscribe send an email to sssd-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
--
_______________________________________________
sssd-users mailing list -- sssd-users@lists.fedorahosted.org
To unsubscribe send an email to sssd-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/sssd-users@lists.fedorahosted.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
Alejandro