Weird issue with "Couldn't invalidate user" - SSSD-AD

[Kodiak Firesmith]

Hello,
I've begun to see the oddest thing within our AD environment on Linux clients (Ubuntu 20, 22).

During logins I see "groups: cannot find name for group ID".

Then during various operations (eg when installing a package that has scripts that create local users, such as postgresql) I see a few of the same userIDs listed as terminal output like this:

Couldn't invalidate user [e-mail]
Couldn't invalidate user [e-mail]
Couldn't invalidate user [e-mail]

Reading through what little comes up in Google for 'Couldn't invalidate user' + sssd, I found old bugs about not being able to invalidate groups in the sss_cache. That got me far enough to have a repeatable action to force this output:

# sss_cache -UG
Couldn't invalidate user [e-mail]
Couldn't invalidate user [e-mail]
Couldn't invalidate user [e-mail]

I've tried ramping up debugging on my AD domain entry in sssd.conf to 9 but I'm not seeing anything that jumps out.

Any ideas?

Thanks!

[Alexey Tikhonov]

Hi,

On Tue, Jan 30, 2024 at 11:22 PM Kodiak Firesmith 
wrote:
> Hello,
> I've begun to see the oddest thing within our AD environment on Linux
> clients (Ubuntu 20, 22).
>
> During logins I see "groups: cannot find name for group ID".
>
> Then during various operations (eg when installing a package that has
> scripts that create local users, such as postgresql) I see a few of the
> same userIDs listed as terminal output like this:
>
> Couldn't invalidate user [e-mail]
> Couldn't invalidate user [e-mail]
> Couldn't invalidate user [e-mail]
>
> Reading through what little comes up in Google for 'Couldn't invalidate
> user' + sssd, I found old bugs about not being able to invalidate groups in
> the sss_cache.  That got me far enough to have a repeatable action to force
> this output:
>
> # sss_cache -UG
> Couldn't invalidate user [e-mail]
> Couldn't invalidate user [e-mail]
> Couldn't invalidate user [e-mail]

Looks like the cache got into an inconsistent state.
Try to delete cache altogether: stop SSSD; rm -rf /var/lib/sss/db/* (or
make a backup if you'd like to); start SSSD

Btw, SSSD shipped in recent Fedora is built without support of handling
local users and for this reason shadow-utils doesn't execute 'sss_cache'
when manipulating local users. Ubuntu should do the same.
> I've tried ramping up debugging on my AD domain entry in sssd.conf to 9
> but I'm not seeing anything that jumps out.
>
> Any ideas?
>
> Thanks!
>
> --
> _______________________________________________
> sssd-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail].o...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue

[Kodiak Firesmith]

Ah yes - I can't believe I didn't do that first before messing with debugging and posting to the list. Absolutely fixed my issue.

I wasn't fully clear on the note about removing support of handling local users so I dug up commit 3e94b64 and have it on my list of things to read through.

Like so many important things, Ubuntu lags pretty hard on SSSD in comparison to RHEL, but I have to assume we'll see the removal down the line sooner or later.

Thanks again for all the help, I'm down to one final SSSD related problem to investigate. If I relent I'll probably post a question about socket activated vs service based responders for pac and nss.

- Kodiak

Sent with [Proton Mail](https://proton.me/) secure email.

On Wednesday, January 31st, 2024 at 3:27 AM, Alexey Tikhonov  wrote:
> Hi,
>
> On Tue, Jan 30, 2024 at 11:22 PM Kodiak Firesmith  wrote:
> > Hello,
> > I've begun to see the oddest thing within our AD environment on Linux clients (Ubuntu 20, 22).
> >
> > During logins I see "groups: cannot find name for group ID".
> >
> > Then during various operations (eg when installing a package that has scripts that create local users, such as postgresql) I see a few of the same userIDs listed as terminal output like this:
> >
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> >
> > Reading through what little comes up in Google for 'Couldn't invalidate user' + sssd, I found old bugs about not being able to invalidate groups in the sss_cache. That got me far enough to have a repeatable action to force this output:
> >
> > # sss_cache -UG
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
>
> Looks like the cache got into an inconsistent state.
> Try to delete cache altogether: stop SSSD; rm -rf /var/lib/sss/db/* (or make a backup if you'd like to); start SSSD
>
> Btw, SSSD shipped in recent Fedora is built without support of handling local users and for this reason shadow-utils doesn't execute 'sss_cache' when manipulating local users. Ubuntu should do the same.
> > I've tried ramping up debugging on my AD domain entry in sssd.conf to 9 but I'm not seeing anything that jumps out.
> >
> > Any ideas?
> >
> > Thanks!
> >
> > --
> > _______________________________________________
> > sssd-users mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedorahosted.org/archives/list/[e-mail].o...
> > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Alexey Tikhonov]

On Wed, Jan 31, 2024 at 2:46 PM Kodiak Firesmith 
wrote:
> Ah yes - I can't believe I didn't do that first before messing with
> debugging and posting to the list.  Absolutely fixed my issue.
>
> I wasn't fully clear on the note about removing support of handling local
> users so I dug up commit 3e94b64 and have it on my list of things to read
> through.

No, that's different - https://github.com/SSSD/sssd/releases/tag/2.9.0 :
```
"Files provider" (i.e. id_provider = files) is deprecated and might be
removed in further releases. Those who are interested to keep using it
awhile should configure its build explicitly using --with-files-provider
./configure option. Or consider using "Proxy provider" with proxy_lib_name
= files instead.
```
And then
https://fedoraproject.org/wiki/Changes/SSSDRemoveFilesProvider
> Like so many important things, Ubuntu lags pretty hard on SSSD in
> comparison to RHEL, but I have to assume we'll see the removal down the
> line sooner or later.
>
> Thanks again for all the help, I'm down to one final SSSD related problem
> to investigate.  If I relent I'll probably post a question about socket
> activated vs service based responders for pac and nss.

IIRC, there was an issue that PAC responder is enabled implicitly for
IPA/AD domains, and Debian/Ubuntu also had socket activation enabled by
default.
(see 'man sssd.conf::implicit_pac_responder')
> Kodiak
>
> Sent with Proton Mail https://proton.me/ secure email.
>
> On Wednesday, January 31st, 2024 at 3:27 AM, Alexey Tikhonov <
> [e-mail]> wrote:
>
> Hi,
>
> On Tue, Jan 30, 2024 at 11:22 PM Kodiak Firesmith <
> [e-mail]> wrote:
> > Hello,
> > I've begun to see the oddest thing within our AD environment on Linux
> > clients (Ubuntu 20, 22).
> >
> > During logins I see "groups: cannot find name for group ID".
> >
> > Then during various operations (eg when installing a package that has
> > scripts that create local users, such as postgresql) I see a few of the
> > same userIDs listed as terminal output like this:
> >
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> >
> > Reading through what little comes up in Google for 'Couldn't invalidate
> > user' + sssd, I found old bugs about not being able to invalidate groups in
> > the sss_cache. That got me far enough to have a repeatable action to force
> > this output:
> >
> > # sss_cache -UG
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
> > Couldn't invalidate user [e-mail]
>
> Looks like the cache got into an inconsistent state.
> Try to delete cache altogether: stop SSSD; rm -rf /var/lib/sss/db/* (or
> make a backup if you'd like to); start SSSD
>
> Btw, SSSD shipped in recent Fedora is built without support of handling
> local users and for this reason shadow-utils doesn't execute 'sss_cache'
> when manipulating local users. Ubuntu should do the same.
> > I've tried ramping up debugging on my AD domain entry in sssd.conf to 9
> > but I'm not seeing anything that jumps out.
> >
> > Any ideas?
> >
> > Thanks!
> >
> > --
> > _______________________________________________
> > sssd-users mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedorahosted.org/archives/list/[e-mail].o...
> > Do not reply to spam, report it:
> > https://pagure.io/fedora-infrastructure/new_issue
>
> --
> _______________________________________________
> sssd-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail].o...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue