Automounting homeshares partially stopped working

[Ronald Wimmer]

Some days ago a strange problem struck us. When colleagues access a 
server using an ipa-automounted share from a Windows client they can 
logon to such a server using a Kerberos ticket but they cannot access 
their NFS-automounted home-share anymore. When they log on with 
username/password they can.

When I try the same from my linux client I do not encounter any problems 
(using the same users as in the above scenario).

Where should I start digging in order to find out what is going wrong?

Cheers,
Ronald

[Ronald Wimmer]

When it does not work I can see the following error in the logs:

Jul 18 15:12:49 myservername gssproxy[5592]: (OID: { 1 2 840 113554 1 2 
2 }) Unspecified GSS failure.  Minor code may provide more information, 
No credentials cache found

[Robbie Harwood]

Ronald Wimmer via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
writes:

>  Some days ago a strange problem struck us. When colleagues access a 
>  server using an ipa-automounted share from a Windows client they can 
>  logon to such a server using a Kerberos ticket but they cannot access 
>  their NFS-automounted home-share anymore. When they log on with 
>  username/password they can. 
Maybe a canonicalization or name mapping bug?  I don't really know how
to debug Windows though, sorry.  Maybe some NFS folks can shed some
light.

Thanks,
--Robbie

[Rolf Linder]

Hi all

We've seen the same issue at our site too.
Kerberos SSO logins do not work for (remote) NFS access anymore. We can access the share
when using password login (or after SSO login by using kinit). Any hints would be greatly
appreciated...

Best regards,
Rolf

-----Original Message-----
From: Robbie Harwood <rharwood(a)redhat.com&gt; 
Sent: 19 July 2019 19:44
To: Ronald Wimmer via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org>; FreeIPA
users list <freeipa-users(a)lists.fedorahosted.org&gt;
Cc: Ronald Wimmer <ronaldw(a)ronzo.at&gt;
Subject: [Freeipa-users] Re: Automounting homeshares partially stopped working

Ronald Wimmer via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
writes:

>  Some days ago a strange problem struck us. When colleagues access a 
>  server using an ipa-automounted share from a Windows client they can 
>  logon to such a server using a Kerberos ticket but they cannot access 
>  their NFS-automounted home-share anymore. When they log on with 
>  username/password they can. 
Maybe a canonicalization or name mapping bug?  I don't really know how to debug
Windows though, sorry.  Maybe some NFS folks can shed some light.

Thanks,
--Robbie

[Rob Crittenden]

Rolf Linder via FreeIPA-users wrote:
>  Hi all
>  
>  We've seen the same issue at our site too.
>  Kerberos SSO logins do not work for (remote) NFS access anymore. We can access the share
> when using password login (or after SSO login by using kinit). Any hints would be greatly
> appreciated... 
You'll need to be sure that the user had a TGT in order to do the mount.

rob

>  
>  Best regards,
>  Rolf
>  
>  -----Original Message-----
>  From: Robbie Harwood <rharwood(a)redhat.com&gt; 
>  Sent: 19 July 2019 19:44
>  To: Ronald Wimmer via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org>; FreeIPA
> users list <freeipa-users(a)lists.fedorahosted.org&gt;
>  Cc: Ronald Wimmer <ronaldw(a)ronzo.at&gt;
>  Subject: [Freeipa-users] Re: Automounting homeshares partially stopped working
>  
>  Ronald Wimmer via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
>  writes:
>  
> > Some days ago a strange problem struck us. When colleagues access a 
> > server using an ipa-automounted share from a Windows client they can 
> > logon to such a server using a Kerberos ticket but they cannot access 
> > their NFS-automounted home-share anymore. When they log on with 
> > username/password they can.
>  
>  Maybe a canonicalization or name mapping bug?  I don't really know how to debug
> Windows though, sorry.  Maybe some NFS folks can shed some light.
>  
>  Thanks,
>  --Robbie
>  
>  
>  
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>

[Ronald Wimmer]

On 22.07.19 16:18, Rob Crittenden wrote:
>  Rolf Linder via FreeIPA-users wrote:
> > Hi all
> >
> > We've seen the same issue at our site too.
> > Kerberos SSO logins do not work for (remote) NFS access anymore. We can access the
> share when using password login (or after SSO login by using kinit). Any hints would be
> greatly appreciated...
>  You'll need to be sure that the user had a TGT in order to do the mount. 
On a
Windows machine the user automatically gets one upon login. The 
strange thing is that it always works when I login from my Linux client 
machine and that it does not when I am coming from Windows...

[Rob Crittenden]

Ronald Wimmer wrote:
>  On 22.07.19 16:18, Rob Crittenden wrote:
> > Rolf Linder via FreeIPA-users wrote:
> >> Hi all
> >>
> >> We've seen the same issue at our site too.
> >> Kerberos SSO logins do not work for (remote) NFS access anymore. We
> >> can access the share when using password login (or after SSO login by
> >> using kinit). Any hints would be greatly appreciated...
> > You'll need to be sure that the user had a TGT in order to do the mount.
>  On a Windows machine the user automatically gets one upon login. The
>  strange thing is that it always works when I login from my Linux client
>  machine and that it does not when I am coming from Windows... 
An assumption here since your workflow isn't completely clear but do you
actually have a ticket on the Linux machine after sshing in from
Windows? Sure seems like you don't.

rob

[Rolf Linder]

hi

We've tested various scenarios all ending in more or less the same output: if using
Keberos-Auth to access a remote NFS share it is only working if you access the server
using password authentication (that is providing the password and then having Kerberos
tickets).

Our tested scenarios were:

1. starting from a windows client (providing a TGT) login to linux server (SSH and Sudo
access is perfectly granted by means of Kerberos auth); NFS access is denied; klist shows
no tickets "klist: Credentials cache keyring 'persistent:[uid]:[gid]' not
found")
2. starting from a windows client enforcing password auth (using IP instead of DNS name
for server), all is fine and klist shows all tickets as it was in the past
3. starting from a linux machine (after password auth and having Kerberos tickets) then
connect to another machine, same issue SSH/Sudo auth is OK, NFS share access is not

This is seen for both IDM local users and AD users (windows user is an AD user usually).

Initial ticket list from windows client:

Cached Tickets: (3)

#0>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#1>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:31 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#2>     Client: rlinder @ USPLAB.LOCAL
        Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


Then after a connection there are more tickets added 

#0>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLABLX.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:23:08 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#1>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#2>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:31 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#3>     Client: rlinder @ USPLAB.LOCAL
        Server: host/labidmclient.usplablx.local @ USPLABLX.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40ad0000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize 0x80000
        Start Time: 7/22/2019 16:23:09 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#4>     Client: rlinder @ USPLAB.LOCAL
        Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


 So far we had no luck in identifying the source why this happens (in our LAB and PROD
environment with different AD and IDM servers).

Rolf

-----Original Message-----
From: Rob Crittenden <rcritten(a)redhat.com&gt; 
Sent: 22 July 2019 16:26
To: Ronald Wimmer <ronaldw(a)ronzo.at>; FreeIPA users list
<freeipa-users(a)lists.fedorahosted.org&gt;
Cc: Linder, Rolf <Rolf.Linder(a)united-security-providers.ch&gt;
Subject: Re: [Freeipa-users] Re: Automounting homeshares partially stopped working

Ronald Wimmer wrote:
>  On 22.07.19 16:18, Rob Crittenden wrote:
> > Rolf Linder via FreeIPA-users wrote:
> >> Hi all
> >>
> >> We've seen the same issue at our site too.
> >> Kerberos SSO logins do not work for (remote) NFS access anymore. We 
> >> can access the share when using password login (or after SSO login 
> >> by using kinit). Any hints would be greatly appreciated...
> > You'll need to be sure that the user had a TGT in order to do the mount.
>  On a Windows machine the user automatically gets one upon login. The 
>  strange thing is that it always works when I login from my Linux 
>  client machine and that it does not when I am coming from Windows...
>  
An assumption here since your workflow isn't completely clear but do you actually have
a ticket on the Linux machine after sshing in from Windows? Sure seems like you
don't.

rob

[Rolf Linder]

...okay, to provide some login output here, may it help...


(Kerberos SSO login from windows machine to linux server - not functional in terms of NFS
share access; note permission denied share is mounted, access is denied)
----
Using username "USPLAB\rlinder".
Last login: Mon Jul 22 16:23:10 2019

Could not chdir to home directory /home/usplab.local/rlinder: Permission denied
-bash: /home/usplab.local/rlinder/.bash_profile: Permission denied
-bash-4.2$ klist
klist: Credentials cache keyring 'persistent:1869201150:1869201150' not found
-bash-4.2$
----

(password auth by using IP instead of DNS name from same windows machine to same linux
server)
----
Using username "USPLAB\rlinder".
USPLAB\rlinder(a)192.168.3.77's password:
Last login: Mon Jul 22 16:40:24 2019

 [[e-mail](a)labidmclient ~]$ klist
Ticket cache: KEYRING:persistent:1869201150:krb_ccache_HILZjf8
Default principal: rlinder(a)USPLAB.LOCAL

Valid starting       Expires              Service principal
07/22/2019 16:43:08  07/23/2019 02:43:07  ldap/labdc01.usplab.local(a)USPLAB.LOCAL
        renew until 07/23/2019 16:43:07
07/22/2019 16:43:07  07/23/2019 02:43:07  krbtgt/USPLAB.LOCAL(a)USPLAB.LOCAL
        renew until 07/23/2019 16:43:07
----

Rolf

-----Original Message-----
From: Linder, Rolf 
Sent: 22 July 2019 16:36
To: 'Rob Crittenden' <rcritten(a)redhat.com>; Ronald Wimmer
<ronaldw(a)ronzo.at>; FreeIPA users list
<freeipa-users(a)lists.fedorahosted.org&gt;
Subject: RE: [Freeipa-users] Re: Automounting homeshares partially stopped working

hi

We've tested various scenarios all ending in more or less the same output: if using
Keberos-Auth to access a remote NFS share it is only working if you access the server
using password authentication (that is providing the password and then having Kerberos
tickets).

Our tested scenarios were:

1. starting from a windows client (providing a TGT) login to linux server (SSH and Sudo
access is perfectly granted by means of Kerberos auth); NFS access is denied; klist shows
no tickets "klist: Credentials cache keyring 'persistent:[uid]:[gid]' not
found") 2. starting from a windows client enforcing password auth (using IP instead
of DNS name for server), all is fine and klist shows all tickets as it was in the past 3.
starting from a linux machine (after password auth and having Kerberos tickets) then
connect to another machine, same issue SSH/Sudo auth is OK, NFS share access is not

This is seen for both IDM local users and AD users (windows user is an AD user usually).

Initial ticket list from windows client:

Cached Tickets: (3)

#0>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#1>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:31 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#2>     Client: rlinder @ USPLAB.LOCAL
        Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


Then after a connection there are more tickets added 

#0>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLABLX.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:23:08 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#1>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#2>     Client: rlinder @ USPLAB.LOCAL
        Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
name_canonicalize
        Start Time: 7/22/2019 16:21:31 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#3>     Client: rlinder @ USPLAB.LOCAL
        Server: host/labidmclient.usplablx.local @ USPLABLX.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40ad0000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize 0x80000
        Start Time: 7/22/2019 16:23:09 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


#4>     Client: rlinder @ USPLAB.LOCAL
        Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
name_canonicalize
        Start Time: 7/22/2019 16:21:58 (local)
        End Time:   7/23/2019 2:21:31 (local)
        Renew Time: 7/29/2019 16:21:31 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96


 So far we had no luck in identifying the source why this happens (in our LAB and PROD
environment with different AD and IDM servers).

Rolf

-----Original Message-----
From: Rob Crittenden <rcritten(a)redhat.com&gt;
Sent: 22 July 2019 16:26
To: Ronald Wimmer <ronaldw(a)ronzo.at>; FreeIPA users list
<freeipa-users(a)lists.fedorahosted.org&gt;
Cc: Linder, Rolf <Rolf.Linder(a)united-security-providers.ch&gt;
Subject: Re: [Freeipa-users] Re: Automounting homeshares partially stopped working

Ronald Wimmer wrote:
>  On 22.07.19 16:18, Rob Crittenden wrote:
> > Rolf Linder via FreeIPA-users wrote:
> >> Hi all
> >>
> >> We've seen the same issue at our site too.
> >> Kerberos SSO logins do not work for (remote) NFS access anymore. We 
> >> can access the share when using password login (or after SSO login 
> >> by using kinit). Any hints would be greatly appreciated...
> > You'll need to be sure that the user had a TGT in order to do the mount.
>  On a Windows machine the user automatically gets one upon login. The 
>  strange thing is that it always works when I login from my Linux 
>  client machine and that it does not when I am coming from Windows...
>  
An assumption here since your workflow isn't completely clear but do you actually have
a ticket on the Linux machine after sshing in from Windows? Sure seems like you
don't.

rob

[Alexander Bokovoy]

On ma, 22 heinä 2019, Rolf Linder via FreeIPA-users wrote:
> hi
>
> We've tested various scenarios all ending in more or less the same
> output: if using Keberos-Auth to access a remote NFS share it is only
> working if you access the server using password authentication (that is
> providing the password and then having Kerberos tickets).
>
> Our tested scenarios were:
>
> 1. starting from a windows client (providing a TGT) login to linux
> server (SSH and Sudo access is perfectly granted by means of Kerberos
> auth); NFS access is denied; klist shows no tickets "klist: Credentials
> cache keyring 'persistent:[uid]:[gid]' not found") 
By default, there is
no TGT delegation from Windows client to Linux
server, thus you don't get the TGT on the Linux side and thus NFS client
cannot request a service ticket to nfs/.. service principal.

As there is no TGT delegation, this is completely expected.

> 2. starting from a windows client enforcing password auth (using IP
> instead of DNS name for server), all is fine and klist shows all
> tickets as it was in the past 
Yes, at this point you are doing actual TGT acquisition on Linux server
side by help of SSSD backend to your PAM authentication in sshd. This is
also expected.

> 3. starting from a linux machine (after password auth and having
> Kerberos tickets) then connect to another machine, same issue SSH/Sudo
> auth is OK, NFS share access is not 
Same as (1), TGT delegation does not happen, thus not possible to obtain
tickets to other services on the Linux server.

You need to decide for your environment whether you are allowing TGT
delegation from the client side to the Linux server. This is typically
controlled by the client itself.

For example, on Linux systems, if you are using 'ssh' client tool, 
'ssh -K' would enable GSSAPI delegation (forwarding) of the credentials
from the client side. Corresponding ssh_config option is

     GSSAPIDelegateCredentials
             Forward (delegate) credentials to the server.  The default is no.

On Windows side it depends on the SSH client at use. In PuTTY it is
controlled this way:
https://documentation.help/PuTTY/config-ssh-auth-gssapi-delegation.html

There are two other methods that are applied by coordinating activities
of the service that wants to use another service in the name of a user
and Kerberos KDC(s) that control both services and users. For that
configuration to be useful, you need to trust the service that obtains a
ticket to NFS service in lieu of the user and the application which
represents that original service has to know how to perform S4U2Proxy
operation. I don't think NFS client actually is capable to do that on
Linux. We use the latter approach in IPA framework -- when you connect
to IPA API end-point (with ipa CLI or Web UI), it is trusted to obtain
Kerberos ticket on your behalf to IPA LDAP server.

If you want concise overview of how it all works from Windows side, see
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-authsod/c...



> This is seen for both IDM local users and AD users (windows user is an AD user usually).
>
> Initial ticket list from windows client:
>
> Cached Tickets: (3)
>
> #0>     Client: rlinder @ USPLAB.LOCAL
>         Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
> name_canonicalize
>         Start Time: 7/22/2019 16:21:58 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #1>     Client: rlinder @ USPLAB.LOCAL
>         Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
> name_canonicalize
>         Start Time: 7/22/2019 16:21:31 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #2>     Client: rlinder @ USPLAB.LOCAL
>         Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
> name_canonicalize
>         Start Time: 7/22/2019 16:21:58 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> Then after a connection there are more tickets added
>
> #0>     Client: rlinder @ USPLAB.LOCAL
>         Server: krbtgt/USPLABLX.LOCAL @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
>         Start Time: 7/22/2019 16:23:08 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #1>     Client: rlinder @ USPLAB.LOCAL
>         Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent
> name_canonicalize
>         Start Time: 7/22/2019 16:21:58 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #2>     Client: rlinder @ USPLAB.LOCAL
>         Server: krbtgt/USPLAB.LOCAL @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent
> name_canonicalize
>         Start Time: 7/22/2019 16:21:31 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #3>     Client: rlinder @ USPLAB.LOCAL
>         Server: host/labidmclient.usplablx.local @ USPLABLX.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40ad0000 -> forwardable renewable pre_authent ok_as_delegate
> name_canonicalize 0x80000
>         Start Time: 7/22/2019 16:23:09 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
> #4>     Client: rlinder @ USPLAB.LOCAL
>         Server: cifs/labDC01.usplab.local @ USPLAB.LOCAL
>         KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
>         Ticket Flags 0x40a50000 -> forwardable renewable pre_authent ok_as_delegate
> name_canonicalize
>         Start Time: 7/22/2019 16:21:58 (local)
>         End Time:   7/23/2019 2:21:31 (local)
>         Renew Time: 7/29/2019 16:21:31 (local)
>         Session Key Type: AES-256-CTS-HMAC-SHA1-96
>
>
>  So far we had no luck in identifying the source why this happens (in our LAB and PROD
> environment with different AD and IDM servers).
>
> Rolf
>
> -----Original Message-----
> From: Rob Crittenden <rcritten(a)redhat.com&gt;
> Sent: 22 July 2019 16:26
> To: Ronald Wimmer <ronaldw(a)ronzo.at>; FreeIPA users list
> <freeipa-users(a)lists.fedorahosted.org&gt;
> Cc: Linder, Rolf <Rolf.Linder(a)united-security-providers.ch&gt;
> Subject: Re: [Freeipa-users] Re: Automounting homeshares partially stopped working
>
> Ronald Wimmer wrote:
> > On 22.07.19 16:18, Rob Crittenden wrote:
> >> Rolf Linder via FreeIPA-users wrote:
> >>> Hi all
> >>>
> >>> We've seen the same issue at our site too.
> >>> Kerberos SSO logins do not work for (remote) NFS access anymore. We
> >>> can access the share when using password login (or after SSO login
> >>> by using kinit). Any hints would be greatly appreciated...
> >> You'll need to be sure that the user had a TGT in order to do the mount.
> > On a Windows machine the user automatically gets one upon login. The
> > strange thing is that it always works when I login from my Linux
> > client machine and that it does not when I am coming from Windows...
>
> An assumption here since your workflow isn't completely clear but do you actually have
> a ticket on the Linux machine after sshing in from Windows? Sure seems like you
> don't.
>
> rob 


> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]... 

-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Rolf Linder]

Thanks very much for this information!

we have configured our client (putty) configuration with these options:

connection => SSH => auth => GSSAPI:

attempt GSSAPI auth: enabled
attempt GSSAPI key exchange: enabled
allow GSSAPI credential delegation: enabled

this is our setup we used in the past which worked fine up to now...

I assume the only reason why this is not working anymore then is to be the
Windows/AD/Putty side of this setup, is that assumption correct from your point of view?

[Alexander Bokovoy]

On ma, 22 heinä 2019, Rolf Linder via FreeIPA-users wrote:
> Thanks very much for this information!
>
> we have configured our client (putty) configuration with these options:
>
> connection => SSH => auth => GSSAPI:
>
> attempt GSSAPI auth: enabled
> attempt GSSAPI key exchange: enabled
> allow GSSAPI credential delegation: enabled
>
> this is our setup we used in the past which worked fine up to now...
>
> I assume the only reason why this is not working anymore then is to be
> the Windows/AD/Putty side of this setup, is that assumption correct
> from your point of view? 
It might be related to a recent update:
https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegatio...


-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Ronald Wimmer]

On 22.07.19 17:29, Alexander Bokovoy wrote:
>  [...] It might be related to a recent update:
>
> https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegatio...
>
>  
I bet we have been struck by that. PuTTY-Settings are correct, 
"Trusted-For-Delegation" is set on the target host and the whole setup 
has been working properly for months.

After reading the above Microsoft-Support article it is still not 
completely clear what needs to be done to make everything work as it did 
before. Can global trust settings be modified in order to resolve this 
issue?

Cheers,
Ronald

PS: I am on holiday for the next two weeks so I won't be able to try 
things out in that time.

[Alexander Bokovoy]

On ti, 23 heinä 2019, Ronald Wimmer wrote:
> On 22.07.19 17:29, Alexander Bokovoy wrote:
> >[...] It might be related to a recent update:
> >https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegation-across-incoming-trusts-in-windows-server
> >
>
> I bet we have been struck by that. PuTTY-Settings are correct, 
> "Trusted-For-Delegation" is set on the target host and the whole setup 
> has been working properly for months.
>
> After reading the above Microsoft-Support article it is still not 
> completely clear what needs to be done to make everything work as it 
> did before. Can global trust settings be modified in order to resolve 
> this issue? 
I have no answer for that. Someone has to try and report.

-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Ronald Wimmer]

On 23.07.19 09:54, Alexander Bokovoy wrote:
>  On ti, 23 heinä 2019, Ronald Wimmer wrote:
> > On 22.07.19 17:29, Alexander Bokovoy wrote:
> >> [...] It might be related to a recent update:
> >>
> https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegatio...
>
> >>
> >>
> >
> > I bet we have been struck by that. PuTTY-Settings are correct, 
> > "Trusted-For-Delegation" is set on the target host and the whole 
> > setup has been working properly for months.
> >
> > After reading the above Microsoft-Support article it is still not 
> > completely clear what needs to be done to make everything work as it 
> > did before. Can global trust settings be modified in order to resolve 
> > this issue?
>  I have no answer for that. Someone has to try and report. 
I was thinking it over again. If it were a global trust setting Ticket 
delegation would also not work from my Linux client. Each windows 
machine used to connect to an IPA server has to allow delegation. Maybe 
that can be done with a policy? Any windows professionals here?

[Rolf Linder]

Some news from our side: 

after latest information provided by Alexander/Rob (thank you very much) we've tested
again and we can confirm within idm (linux) all is fine (if using forward delegation it
works). 
From windows to linux it stopped working and so we've asked our windows admin
colleagues and they've rollbacked the latest updates for the past couple days in our
LAB.

As it seems (at least) KB4507448 [1] update breaks the out-of-box support for AD-User
accessing remote NFS-share using kerberized (forwarded) access. 

From our point of view, the workaround for now will be (in accordance to [2]) to configure
an "insecure" domain setting on AD side. If possible this could be eliminated to
not have to document an not secure setting in the future...

Once more, thank you both Alexander/Rob in participating and helping to solve the core
issue.

Rolf

[1] https://support.microsoft.com/en-us/help/4507448/windows-8-1-update-kb450...
[2]
https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegatio...

[Ronald Wimmer]

Unfortunately, when I try the following on an affected windows machine 
it does not work:

C:\> netdom trust linux.mydomain.at /domain:ad.mydomain.at 
/EnableTGTDelegation:
Yes /verbose
Establishing a session with \\ipa1.linux.mydomain.at

Reading LSA domain policy information

Deleting the session with \\ipa1.linux.mydomain.at

Access is denied.

The command failed to complete successfully.

Am I using netdom the right way? What are your thoughts?

Cheers,
Ronald

[Alexander Bokovoy]

On ti, 23 heinä 2019, Ronald Wimmer via FreeIPA-users wrote:
> Unfortunately, when I try the following on an affected windows machine
>
> it does not work:
>
> C:\> netdom trust linux.mydomain.at /domain:ad.mydomain.at 
> /EnableTGTDelegation:
> Yes /verbose
> Establishing a session with \\ipa1.linux.mydomain.at
>
> Reading LSA domain policy information
>
> Deleting the session with \\ipa1.linux.mydomain.at
>
> Access is denied.
>
> The command failed to complete successfully.
>
> Am I using netdom the right way? What are your thoughts? 
I think it should be the
other way around, e.g.

netdom trust ad.mydomain.at /domain:linux.mydomain.at /enabletgtdelegation:Yes /verbose

-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Ronald Wimmer]

On 23.07.19 15:19, Alexander Bokovoy wrote:
>  netdom trust ad.mydomain.at /domain:linux.mydomain.at 
>  /enabletgtdelegation:Yes /verbose  
I think you are right. This way a domain controller is contacted. But 
still, I get an "Access is denied.". Presumably I would need an AD admin 
to issue that particular command...

[Alexander Bokovoy]

On ti, 23 heinä 2019, Ronald Wimmer wrote:
> On 23.07.19 15:19, Alexander Bokovoy wrote:
> >netdom trust ad.mydomain.at /domain:linux.mydomain.at 
> >/enabletgtdelegation:Yes /verbose
>
> I think you are right. This way a domain controller is contacted. But 
> still, I get an "Access is denied.". Presumably I would need an AD 
> admin to issue that particular command... 
Yes, you need credentials of a someone who can modify forest trusts.

-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Ronald Wimmer]

On 23.07.19 16:03, Alexander Bokovoy wrote:
>  On ti, 23 heinä 2019, Ronald Wimmer wrote:
> > On 23.07.19 15:19, Alexander Bokovoy wrote:
> >> netdom trust ad.mydomain.at /domain:linux.mydomain.at 
> >> /enabletgtdelegation:Yes /verbose
> >
> > I think you are right. This way a domain controller is contacted. But 
> > still, I get an "Access is denied.". Presumably I would need an AD 
> > admin to issue that particular command...
>
>  Yes, you need credentials of a someone who can modify forest trusts.
>  
I delegated that task to our AD guys. We will see if the issue is 
resolved afterwards.

[Ronald Wimmer]

On 23.07.19 16:06, Ronald Wimmer via FreeIPA-users wrote:
>  On 23.07.19 16:03, Alexander Bokovoy wrote:
> > On ti, 23 heinä 2019, Ronald Wimmer wrote:
> >> On 23.07.19 15:19, Alexander Bokovoy wrote:
> >>> netdom trust ad.mydomain.at /domain:linux.mydomain.at 
> >>> /enabletgtdelegation:Yes /verbose
> >>
> >> I think you are right. This way a domain controller is contacted. 
> >> But still, I get an "Access is denied.". Presumably I would need an 
> >> AD admin to issue that particular command...
> >
> > Yes, you need credentials of a someone who can modify forest trusts.
> >
>  I delegated that task to our AD guys. We will see if the issue is 
>  resolved afterwards.
>  
After issuing the netdom command TGT delegation is working again. Thank 
you very much for your valuable input Mr. Bokovoy!

Cheers,
Ronald

[Ronald Wimmer]

On 22.07.19 17:29, Alexander Bokovoy wrote:
>  [...] It might be related to a recent update:
>
> https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegatio...
>
>  
If i try to issue the command

netdom trust second.mydomain.at /domain:linux.mydomain.at 
/enabletgtdelegation:Yes /verbose

for a second trusted domain I get an error message saying that user or 
password is wrong. According to IPA the trust type is "Non-transitive 
external trust to a domain in another Active Directory forest".

Any ideas?

Cheers,
Ronald

[Alexander Bokovoy]

On ma, 19 elo 2019, Ronald Wimmer wrote:
> On 22.07.19 17:29, Alexander Bokovoy wrote:
> >[...] It might be related to a recent update:
> >https://support.microsoft.com/en-us/help/4490425/updates-to-tgt-delegation-across-incoming-trusts-in-windows-server
> >
>
> If i try to issue the command
>
> netdom trust second.mydomain.at /domain:linux.mydomain.at /enabletgtdelegation:Yes
> /verbose
>
> for a second trusted domain I get an error message saying that user or 
> password is wrong. According to IPA the trust type is "Non-transitive 
> external trust to a domain in another Active Directory forest".
>
> Any ideas? 
So, IPA is in external trust to AD?

Note that communication to IPA DC using netdom tools most likely will
not work until we get all the fixes I'm working on right now in the
release. *Some* of them are in Fedora 30 updates already and some of
them will be in upcoming RHEL 8 minor releases. But anything before that
will still have not working netdom against IPA DCs.

So I would say this is as it should be.

However, for the purpose of this thread, we should not even try to
contact IPA DCs at all. The change should happen on AD DC side.


-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[Ronald Wimmer]

On 22.07.19 16:25, Rob Crittenden wrote:
>  [...]
>  An assumption here since your workflow isn't completely clear but do you
>  actually have a ticket on the Linux machine after sshing in from
>  Windows? Sure seems like you don't. 
The affected users do not have any Kerberos ticket on the target machine.

Another strange thing. When I login to the same machine, I can see and 
access the NFS share although klist shows no tickets:

  klist
klist: Credentials cache keyring 
'persistent:1073895519:krb_ccache_e9F1GDY' not found