Potentially false errors when changing user password via `passwd` using krb5 as the chpass_provider

[Jesse Szwedko]

Hi all,

We are seeing some strange behavior when using passwd (via sssd using the
krb5 password provider) to change a user's password where the command
reports that the change fails due to "Authentication token manipulation
error" every other time (meaning, it will error, then not error, then
error, then not error, and so forth, in sequence). However, the operation
is actually successful (in that it changes the password) even when it
reports the error.

We saw this behavior with sssd versions 1.12.2 and 1.13.3 (we tried
upgrading to see if maybe the issue had been addressed).

Shell output executing passwd:
[test-user@ip-172-31-44-254 ~]$ passwd
Changing password for user test-user.
Current Password:
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[test-user@ip-172-31-44-254 ~]$ passwd
Changing password for user test-user.
Current Password:
New password:
Retype new password:
passwd: Authentication token manipulation error

Our sssd.conf: http://ix.io/neC
SSSD logs: http://ix.io/neq

The logs show the use of `passwd` twice, where the first time did not
report the error, and the second did. You'll note that when `passwd`
reported the error, that sssd logged "9027 (Wed Jan  6 21:08:57 2016)
[[sssd[krb5_child[31229]]]] [sss_child_krb5_trace_cb] (0x4000): [31229]
1452114537.218562: Received error from KDC: -1765328360/Preauthentication
failed".

Using `kpasswd` directly does not exhibit this behavior, i.e. it changes
the password and reports success in all instances where we've tried it.

The KDC we are authenticating against is managed service from Amazon Web
services, Simple AD, which feels like it is Samba (somewhat corroborated by
this answer on ServerFault:
http://serverfault.com/questions/746194/is-it-possible-to-use-kerberos-ov...
which states "In addition, Amazon's Simple AD is Samba AD built with
Heimdal kerberos"). We do not have a lot of insight into the configuration
of the server, but can query it via LDAP and kerberos methods if any
additional information would be useful.

This isn't a blocking issue for us since the password change is actually
successful, but it is somewhat of a curiosity and loose end I was hoping
someone could assist with.

Thank you!
-Jesse

[Jakub Hrozek]

On Wed, Jan 06, 2016 at 03:08:50PM -0800, Jesse Szwedko wrote:
>  Hi all,
>  
>  We are seeing some strange behavior when using passwd (via sssd using the
>  krb5 password provider) to change a user's password where the command
>  reports that the change fails due to "Authentication token manipulation
>  error" every other time (meaning, it will error, then not error, then
>  error, then not error, and so forth, in sequence). However, the operation
>  is actually successful (in that it changes the password) even when it
>  reports the error.
>  
>  We saw this behavior with sssd versions 1.12.2 and 1.13.3 (we tried
>  upgrading to see if maybe the issue had been addressed).
>  
>  Shell output executing passwd:
>  [test-user@ip-172-31-44-254 ~]$ passwd
>  Changing password for user test-user.
>  Current Password:
>  New password:
>  Retype new password:
>  passwd: all authentication tokens updated successfully.
>  [test-user@ip-172-31-44-254 ~]$ passwd
>  Changing password for user test-user.
>  Current Password:
>  New password:
>  Retype new password:
>  passwd: Authentication token manipulation error
>  
>  Our sssd.conf: http://ix.io/neC
>  SSSD logs: http://ix.io/neq
>  
>  The logs show the use of `passwd` twice, where the first time did not
>  report the error, and the second did. You'll note that when `passwd`
>  reported the error, that sssd logged "9027 (Wed Jan  6 21:08:57 2016)
>  [[sssd[krb5_child[31229]]]] [sss_child_krb5_trace_cb] (0x4000): [31229]
>  1452114537.218562: Received error from KDC: -1765328360/Preauthentication
>  failed". 
Preauthentication failed means wrong password. I would guess that
acquiring a TGT with the new password had failed..does klist show a TGT
after the failed attempt?

Also that would explain why kpasswd works, it doesn't acquire a TGT
using the new creds.

However, since you're using the same server for authentication and
password change, I'm not sure why the kinit after the password change
would fail. Can you look into the non-redacted logs if both the action
with kadmin/changepw and the subsequent get_and_save_tgt() talk to the
same server?