SSSD login stopped working on Ubuntu 22.04

[Joyce Babu]

I have a FreeIPA installation with many Pop!_OS 21.10 clients. Today I upgraded one of the
clients to Pop!_OS 22.04, and I can no longer authenticate with FreeIPA on the upgraded
client.

In krb5kdc.log file on the server, I can see the error 'verify failure: Incorrect
password in encrypted challenge'

=======
May 17 14:07:43 ipa.myhost.com krb5kdc[301](info): AS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
192.168.10.14: NEEDED_PREAUTH: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
Additional pre-authentication required
May 17 14:07:43 ipa.myhost.com krb5kdc[301](info): closing down fd 12
May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): preauth (encrypted_challenge) verify
failure: Incorrect password in encrypted challenge
May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): AS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
192.168.10.14: PREAUTH_FAILED: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
Preauthentication failed
May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): closing down fd 12
=======

If I try the same username/password on a Pop!_OS 21.10 client, I can login successfully
and I see the following log message. I tried multiple times with multiple users, and had
the same result.

=======
May 17 14:05:51 ipa.myhost.com krb5kdc[299](info): AS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
192.168.10.24: NEEDED_PREAUTH: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
Additional pre-authentication required
May 17 14:05:51 ipa.myhost.com krb5kdc[299](info): closing down fd 12
May 17 14:05:51 ipa.myhost.com krb5kdc[301](info): AS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
192.168.10.24: ISSUE: authtime 1652796351, etypes {rep=aes256-cts-hmac-sha1-96(18),
tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, joyce(a)MYHOST.COM for
krbtgt/MYHOST.COM(a)MYHOST.COM
May 17 14:05:51 ipa.myhost.com krb5kdc[301](info): closing down fd 12
May 17 14:05:51 ipa.myhost.com krb5kdc[300](info): TGS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
192.168.10.24: ISSUE: authtime 1652796351, etypes {rep=aes256-cts-hmac-sha1-96(18),
tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, joyce(a)MYHOST.COM for
host/ws024.office-mng.myhost.net(a)MYHOST.COM
May 17 14:05:51 ipa.myhost.com krb5kdc[300](info): closing down fd 12
=======

What changed in Ubuntu 22.04? Could this be due to incompatible encryption type?

[Sumit Bose]

Am Tue, May 17, 2022 at 02:29:24PM -0000 schrieb Joyce Babu via FreeIPA-users:
>  I have a FreeIPA installation with many Pop!_OS 21.10 clients. Today
> I upgraded one of the clients to Pop!_OS 22.04, and I can no longer authenticate with
> FreeIPA on the upgraded client.
>  
>  In krb5kdc.log file on the server, I can see the error 'verify failure: Incorrect
> password in encrypted challenge'
>  
>  =======
>  May 17 14:07:43 ipa.myhost.com krb5kdc[301](info): AS_REQ (8 etypes
> {aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
> aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
> DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
> 192.168.10.14: NEEDED_PREAUTH: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
> Additional pre-authentication required
>  May 17 14:07:43 ipa.myhost.com krb5kdc[301](info): closing down fd 12
>  May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): preauth (encrypted_challenge) verify
> failure: Incorrect password in encrypted challenge
>  May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): AS_REQ (8 etypes
> {aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
> aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
> DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
> 192.168.10.14: PREAUTH_FAILED: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
> Preauthentication failed
>  May 17 14:07:43 ipa.myhost.com krb5kdc[302](info): closing down fd 12
>  =======
>  
>  If I try the same username/password on a Pop!_OS 21.10 client, I can login successfully
> and I see the following log message. I tried multiple times with multiple users, and had
> the same result.
>  
>  =======
>  May 17 14:05:51 ipa.myhost.com krb5kdc[299](info): AS_REQ (8 etypes
> {aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
> aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
> DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
> 192.168.10.24: NEEDED_PREAUTH: joyce(a)MYHOST.COM for krbtgt/MYHOST.COM(a)MYHOST.COM,
> Additional pre-authentication required
>  May 17 14:05:51 ipa.myhost.com krb5kdc[299](info): closing down fd 12
>  May 17 14:05:51 ipa.myhost.com krb5kdc[301](info): AS_REQ (8 etypes
> {aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
> aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
> DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
> 192.168.10.24: ISSUE: authtime 1652796351, etypes {rep=aes256-cts-hmac-sha1-96(18),
> tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, joyce(a)MYHOST.COM for
> krbtgt/MYHOST.COM(a)MYHOST.COM
>  May 17 14:05:51 ipa.myhost.com krb5kdc[301](info): closing down fd 12
>  May 17 14:05:51 ipa.myhost.com krb5kdc[300](info): TGS_REQ (8 etypes
> {aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
> aes128-cts-hmac-sha256-128(19), UNSUPPORTED:des3-hmac-sha1(16),
> DEPRECATED:arcfour-hmac(23), camellia128-cts-cmac(25), camellia256-cts-cmac(26)})
> 192.168.10.24: ISSUE: authtime 1652796351, etypes {rep=aes256-cts-hmac-sha1-96(18),
> tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, joyce(a)MYHOST.COM for
> host/ws024.office-mng.myhost.net(a)MYHOST.COM
>  May 17 14:05:51 ipa.myhost.com krb5kdc[300](info): closing down fd 12
>  =======
>  
>  What changed in Ubuntu 22.04? Could this be due to incompatible encryption type? 
Hi,

have you checked if the keyboard encoding changed and you have to type
the special characters of the password differently now?

bye,
Sumit

>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Joyce Babu]

Thank you for your response.

The password I entered is alpha numeric with no special characters. Also, I tried to login
to both the old and new client through SSH from my laptop. So, it is not a keyboard
encoding issue.

[Sumit Bose]

Am Tue, May 17, 2022 at 08:22:30PM -0000 schrieb Joyce Babu via FreeIPA-users:
>  Thank you for your response.
>  
>  The password I entered is alpha numeric with no special characters. Also, I tried to
> login to both the old and new client through SSH from my laptop. So, it is not a keyboard
> encoding issue. 
Hi,

can you set 'debug_level = 9' in the [domain/...] section of sssd.conf,
restart SSSD, try to log in again and then send the logs from
/var/log/sssd/ ?

bye,
Sumit

>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Joyce Babu]

Hello Sumit,

I have generated the logs files.

Is it okay, if I email the files directly to you?

*Thanks and regards,*
Joyce Babu

[Sumit Bose]

Am Tue, May 24, 2022 at 07:45:01PM +0530 schrieb Joyce Babu via FreeIPA-users:
>  Hello Sumit,
>  
>  I have generated the logs files.
>  
>  Is it okay, if I email the files directly to you? 
Hi,

sure

bye,
Sumit

>  
>  *Thanks and regards,*
>  Joyce Babu 
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Joyce Babu]

Thanks to Sumit's help, I was finally able to identify the cause of the
login failure.

The default shell for the user was set to /usr/bin/zsh on the FreeIPA
server. But since zsh is not installed by default on a new Pop OS
installation, sshd was sending a dummy password to sssd. Installing zsh
resolved the issue for me.

In case someone else finds it useful, I am sharing the comment from Sumit
that helped identify the problem.

thanks, it looks like this is a feature of sshd. If sshd thinks the user
>  is invalid or not allowed to log in it sends a dummy password
> looking
>  like the one from the strace to mitigate timing attacks. 

>  It looks like you are trying to log in with the short name
>  '<redacted_username>'. Can you send the output of 

>      getent passwd <redacted_username>