Gssproxy stops looking for Apache ccache after awhile

[Wen Liang]

My CentOS 7.2 server:

* Enrolled IPA client
* Automount configured with NFSv4 krb5 mount for user home directories.

Everything works.  IPA users can login and their home directory would automatically be mounted.

Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed

https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).

And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:

gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)

gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)

What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):

gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)

But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.

So why is it not using service "apache" all the time and instead falls back to "nfs-client"?

[Simo Sorce]

On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> My CentOS 7.2 server:
> Enrolled IPA client
> Automount configured with NFSv4 krb5 mount for user home directories.
>
> Everything works.  IPA users can login and their home directory would automatically be mounted.
>
> Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
>
> https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
>
> And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
>
> gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
>
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
>
> What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
>
> gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
>
> But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
>
> So why is it not using service "apache" all the time and instead falls back to "nfs-client"?

The nfs-client (rpc.gssd I assume here) acquires credentials with the
nfs (or host) keytab to connect to the remote server for non-user
related operations.

I think it is unrelated to your problem with apache, do you see any
error returned by gssproxy when apache stops getting access ?
What is the nfs server ?

Simo.

[Wen Liang]

The NFS server is also running Centos 7.2 and an IPA client.

When Apache starts returning "no permission" error, gssproxy would log the messages that I posted earlier (for service "nfs-client").  Also I don't see any krb5cc file created in /var/lib/gssproxy/clients/.  Whereas if I remove /var/lib/gssproxy/clients/krb5cc_apache, restart gssproxy, and hit the URL that maps to an user's home directory, I would see the file /var/lib/gssproxy/clients/krb5cc_apache get created, as expected according to the gssproxy.conf.

Here is /etc/gssproxy/gssproxy.conf:

[gssproxy] 
debug = yes

[service/apache] 
mechs = krb5 
cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_apache 
cred_store = client_keytab:/var/lib/gssproxy/clients/http.keytab 
cred_usage = initiate 
euid = 48

[service/nfs-server] 
mechs = krb5 
socket = /run/gssproxy.sock 
cred_store = keytab:/etc/krb5.keytab 
trusted = yes 
kernel_nfsd = yes 
euid = 0

[service/nfs-client] 
mechs = krb5 
cred_store = keytab:/etc/krb5.keytab 
cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U 
cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab 
cred_usage = initiate 
allow_any_uid = yes 
trusted = yes 
euid = 0

On Thursday, February 25, 2016 1:38 PM, Simo Sorce  wrote:
On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> My CentOS 7.2 server:
> Enrolled IPA client
> Automount configured with NFSv4 krb5 mount for user home directories.
>
> Everything works.  IPA users can login and their home directory would automatically be mounted.
>
> Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
>
> https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
>
> And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
>
> gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
>
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
>
> What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
>
> gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
>
> But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
>
> So why is it not using service "apache" all the time and instead falls back to "nfs-client"?

The nfs-client (rpc.gssd I assume here) acquires credentials with the
nfs (or host) keytab to connect to the remote server for non-user
related operations.

I think it is unrelated to your problem with apache, do you see any
error returned by gssproxy when apache stops getting access ?
What is the nfs server ?

Simo.

[Simo Sorce]

On Thu, 2016-02-25 at 20:21 +0000, Wen Liang wrote:
> The NFS server is also running Centos 7.2 and an IPA client.
>
> When Apache starts returning "no permission" error, gssproxy would log
> the messages that I posted earlier (for service "nfs-client").  Also I
> don't see any krb5cc file created in /var/lib/gssproxy/clients/.
> Whereas if I remove /var/lib/gssproxy/clients/krb5cc_apache, restart
> gssproxy, and hit the URL that maps to an user's home directory, I
> would see the file /var/lib/gssproxy/clients/krb5cc_apache get
> created, as expected according to the gssproxy.conf.

This sounds odd, if you see only euid=0 connecting then it means
rpc.gssd is not accessing gssproxy with the right credentials.

Does the problem also go away if you restart rpc.gssd
(nfs-secure-client) instead of restarting gssproxy ?

Simo.
> Here is /etc/gssproxy/gssproxy.conf:
>
> [gssproxy] 
> debug = yes
>
> [service/apache] 
> mechs = krb5 
> cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_apache 
> cred_store = client_keytab:/var/lib/gssproxy/clients/http.keytab 
> cred_usage = initiate 
> euid = 48
>
> [service/nfs-server] 
> mechs = krb5 
> socket = /run/gssproxy.sock 
> cred_store = keytab:/etc/krb5.keytab 
> trusted = yes 
> kernel_nfsd = yes 
> euid = 0
>
> [service/nfs-client] 
> mechs = krb5 
> cred_store = keytab:/etc/krb5.keytab 
> cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U 
> cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab 
> cred_usage = initiate 
> allow_any_uid = yes 
> trusted = yes 
> euid = 0
> On Thursday, February 25, 2016 1:38 PM, Simo Sorce  wrote:
> On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> > My CentOS 7.2 server:
> > Enrolled IPA client
> > Automount configured with NFSv4 krb5 mount for user home directories.
> >
> > Everything works.  IPA users can login and their home directory would automatically be mounted.
> >
> > Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
> >
> > https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
> >
> > And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
> >
> > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
> >
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
> >
> > What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
> >
> > gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> >
> > But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
> >
> > So why is it not using service "apache" all the time and instead falls back to "nfs-client"?
>
> The nfs-client (rpc.gssd I assume here) acquires credentials with the
> nfs (or host) keytab to connect to the remote server for non-user
> related operations.
>
> I think it is unrelated to your problem with apache, do you see any
> error returned by gssproxy when apache stops getting access ?
> What is the nfs server ?
>
> Simo.

[Wen Liang]

Yes, restarting rpc.gssd (systemctl restart rpcgssd) also solves the Apache permission.

On Thursday, February 25, 2016 3:38 PM, Simo Sorce  wrote:
On Thu, 2016-02-25 at 20:21 +0000, Wen Liang wrote:
> The NFS server is also running Centos 7.2 and an IPA client.
>
> When Apache starts returning "no permission" error, gssproxy would log
> the messages that I posted earlier (for service "nfs-client").  Also I
> don't see any krb5cc file created in /var/lib/gssproxy/clients/.
> Whereas if I remove /var/lib/gssproxy/clients/krb5cc_apache, restart
> gssproxy, and hit the URL that maps to an user's home directory, I
> would see the file /var/lib/gssproxy/clients/krb5cc_apache get
> created, as expected according to the gssproxy.conf.

This sounds odd, if you see only euid=0 connecting then it means
rpc.gssd is not accessing gssproxy with the right credentials.

Does the problem also go away if you restart rpc.gssd
(nfs-secure-client) instead of restarting gssproxy ?

Simo.
> Here is /etc/gssproxy/gssproxy.conf:
>
> [gssproxy] 
> debug = yes
>
> [service/apache] 
> mechs = krb5 
> cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_apache 
> cred_store = client_keytab:/var/lib/gssproxy/clients/http.keytab 
> cred_usage = initiate 
> euid = 48
>
> [service/nfs-server] 
> mechs = krb5 
> socket = /run/gssproxy.sock 
> cred_store = keytab:/etc/krb5.keytab 
> trusted = yes 
> kernel_nfsd = yes 
> euid = 0
>
> [service/nfs-client] 
> mechs = krb5 
> cred_store = keytab:/etc/krb5.keytab 
> cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U 
> cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab 
> cred_usage = initiate 
> allow_any_uid = yes 
> trusted = yes 
> euid = 0
> On Thursday, February 25, 2016 1:38 PM, Simo Sorce  wrote:
> On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> > My CentOS 7.2 server:
> > Enrolled IPA client
> > Automount configured with NFSv4 krb5 mount for user home directories.
> >
> > Everything works.  IPA users can login and their home directory would automatically be mounted.
> >
> > Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
> >
> > https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
> >
> > And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
> >
> > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
> >
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
> >
> > What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
> >
> > gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> >
> > But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
> >
> > So why is it not using service "apache" all the time and instead falls back to "nfs-client"?
>
> The nfs-client (rpc.gssd I assume here) acquires credentials with the
> nfs (or host) keytab to connect to the remote server for non-user
> related operations.
>
> I think it is unrelated to your problem with apache, do you see any
> error returned by gssproxy when apache stops getting access ?
> What is the nfs server ?
>
> Simo.

[Simo Sorce]

On Thu, 2016-02-25 at 20:46 +0000, Wen Liang wrote:
> Yes, restarting rpc.gssd (systemctl restart rpcgssd) also solves the Apache permission.

Ok, his is good information,
I wonder if there is a bug in rpc,gssd or in the gssproxy client.

What version of gss-proxy are you running exactly ?

Simo.
> On Thursday, February 25, 2016 3:38 PM, Simo Sorce  wrote:
> On Thu, 2016-02-25 at 20:21 +0000, Wen Liang wrote:
> > The NFS server is also running Centos 7.2 and an IPA client.
> >
> > When Apache starts returning "no permission" error, gssproxy would log
> > the messages that I posted earlier (for service "nfs-client").  Also I
> > don't see any krb5cc file created in /var/lib/gssproxy/clients/.
> > Whereas if I remove /var/lib/gssproxy/clients/krb5cc_apache, restart
> > gssproxy, and hit the URL that maps to an user's home directory, I
> > would see the file /var/lib/gssproxy/clients/krb5cc_apache get
> > created, as expected according to the gssproxy.conf.
>
> This sounds odd, if you see only euid=0 connecting then it means
> rpc.gssd is not accessing gssproxy with the right credentials.
>
> Does the problem also go away if you restart rpc.gssd
> (nfs-secure-client) instead of restarting gssproxy ?
>
> Simo.
> > Here is /etc/gssproxy/gssproxy.conf:
> >
> > [gssproxy] 
> > debug = yes
> >
> > [service/apache] 
> > mechs = krb5 
> > cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_apache 
> > cred_store = client_keytab:/var/lib/gssproxy/clients/http.keytab 
> > cred_usage = initiate 
> > euid = 48
> >
> > [service/nfs-server] 
> > mechs = krb5 
> > socket = /run/gssproxy.sock 
> > cred_store = keytab:/etc/krb5.keytab 
> > trusted = yes 
> > kernel_nfsd = yes 
> > euid = 0
> >
> > [service/nfs-client] 
> > mechs = krb5 
> > cred_store = keytab:/etc/krb5.keytab 
> > cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U 
> > cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab 
> > cred_usage = initiate 
> > allow_any_uid = yes 
> > trusted = yes 
> > euid = 0
> > On Thursday, February 25, 2016 1:38 PM, Simo Sorce  wrote:
> > On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> > > My CentOS 7.2 server:
> > > Enrolled IPA client
> > > Automount configured with NFSv4 krb5 mount for user home directories.
> > >
> > > Everything works.  IPA users can login and their home directory would automatically be mounted.
> > >
> > > Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
> > >
> > > https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
> > >
> > > And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
> > >
> > > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
> > >
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
> > >
> > > What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
> > >
> > > gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> > > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > >
> > > But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
> > >
> > > So why is it not using service "apache" all the time and instead falls back to "nfs-client"?
> >
> > The nfs-client (rpc.gssd I assume here) acquires credentials with the
> > nfs (or host) keytab to connect to the remote server for non-user
> > related operations.
> >
> > I think it is unrelated to your problem with apache, do you see any
> > error returned by gssproxy when apache stops getting access ?
> > What is the nfs server ?
> >
> > Simo.

[Wen Liang]

Here is the information:

➜  ~ gssproxy --version 
0.4.1 
➜  ~ rpm -qa gssproxy 
gssproxy-0.4.1-7.el7.x86_64

➜  ~ rpm -qa nfs-utils 
nfs-utils-1.3.0-0.21.el7.x86_64

Thanks,
Wen

On Thursday, February 25, 2016 4:00 PM, Simo Sorce  wrote:
On Thu, 2016-02-25 at 20:46 +0000, Wen Liang wrote:
> Yes, restarting rpc.gssd (systemctl restart rpcgssd) also solves the Apache permission.

Ok, his is good information,
I wonder if there is a bug in rpc,gssd or in the gssproxy client.

What version of gss-proxy are you running exactly ?

Simo.
> On Thursday, February 25, 2016 3:38 PM, Simo Sorce  wrote:
> On Thu, 2016-02-25 at 20:21 +0000, Wen Liang wrote:
> > The NFS server is also running Centos 7.2 and an IPA client.
> >
> > When Apache starts returning "no permission" error, gssproxy would log
> > the messages that I posted earlier (for service "nfs-client").  Also I
> > don't see any krb5cc file created in /var/lib/gssproxy/clients/.
> > Whereas if I remove /var/lib/gssproxy/clients/krb5cc_apache, restart
> > gssproxy, and hit the URL that maps to an user's home directory, I
> > would see the file /var/lib/gssproxy/clients/krb5cc_apache get
> > created, as expected according to the gssproxy.conf.
>
> This sounds odd, if you see only euid=0 connecting then it means
> rpc.gssd is not accessing gssproxy with the right credentials.
>
> Does the problem also go away if you restart rpc.gssd
> (nfs-secure-client) instead of restarting gssproxy ?
>
> Simo.
> > Here is /etc/gssproxy/gssproxy.conf:
> >
> > [gssproxy] 
> > debug = yes
> >
> > [service/apache] 
> > mechs = krb5 
> > cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_apache 
> > cred_store = client_keytab:/var/lib/gssproxy/clients/http.keytab 
> > cred_usage = initiate 
> > euid = 48
> >
> > [service/nfs-server] 
> > mechs = krb5 
> > socket = /run/gssproxy.sock 
> > cred_store = keytab:/etc/krb5.keytab 
> > trusted = yes 
> > kernel_nfsd = yes 
> > euid = 0
> >
> > [service/nfs-client] 
> > mechs = krb5 
> > cred_store = keytab:/etc/krb5.keytab 
> > cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U 
> > cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab 
> > cred_usage = initiate 
> > allow_any_uid = yes 
> > trusted = yes 
> > euid = 0
> > On Thursday, February 25, 2016 1:38 PM, Simo Sorce  wrote:
> > On Thu, 2016-02-25 at 15:59 +0000, Wen Liang wrote:
> > > My CentOS 7.2 server:
> > > Enrolled IPA client
> > > Automount configured with NFSv4 krb5 mount for user home directories.
> > >
> > > Everything works.  IPA users can login and their home directory would automatically be mounted.
> > >
> > > Now, I am trying to configure the server's Apache to expose certain users' home directory for web.  To do this, I followed
> > >
> > > https://fedorahosted.org/gss-proxy/wiki/NFS to configure gssproxy (no impersonation).
> > >
> > > And this works... for awhile.  After idling awhile, Apache would report no permission, and gssproxy debug message would show:
> > >
> > > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "nfs-client", euid: 0, socket: (null)
> > >
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "nfs-client", euid: 0, socket: (null)
> > >
> > > What I don't understand is why is it using service "nfs-client"?  Shouldn't it be using service "apache", which is set above the section for "nfs-client" in gssproxy.conf?  I say this because if I restart gssproxy, Apache would work again, and the debug message would show (as expected):
> > >
> > > gssproxy: Client connected (fd = 12) (pid = 3158) (uid = 48) (gid = 48) (context = system_u:system_r:gssd_t:s0)
> > > gssproxy: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "apache", euid: 48, socket: (null)
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > > gssproxy: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "apache", euid: 48, socket: (null)
> > >
> > > But after idling awhile, Apache would again stop working and gssproxy debug messages would show it's using "nfs-client" again.
> > >
> > > So why is it not using service "apache" all the time and instead falls back to "nfs-client"?
> >
> > The nfs-client (rpc.gssd I assume here) acquires credentials with the
> > nfs (or host) keytab to connect to the remote server for non-user
> > related operations.
> >
> > I think it is unrelated to your problem with apache, do you see any
> > error returned by gssproxy when apache stops getting access ?
> > What is the nfs server ?
> >
> > Simo.

[Anthony Joseph Messina]

On Thursday, February 25, 2016 03:59:59 PM Simo Sorce wrote:
> On Thu, 2016-02-25 at 20:46 +0000, Wen Liang wrote:
> > Yes, restarting rpc.gssd (systemctl restart rpcgssd) also solves the
> > Apache permission.
>
> Ok, his is good information,
> I wonder if there is a bug in rpc,gssd or in the gssproxy client.
>
> What version of gss-proxy are you running exactly ?
>
> Simo.

I can confirm this same issue, but I don't use gssproxy for the HTTP service 
keytab, just the regular NFS client/server gssproxy.conf defaults to allow 
Apache to serve KRB5/NFSv4 /home dirs. (Just didn't get a chance to report 
it). -A

[Wen Liang]

Here is a workaround that works for me.  Create a 5 minutes cron job that:
hits an URL for the NFS mount and then restart gssproxy.

Just restarting gssproxy by itself doesn't work.  The permission problem
will eventually come back.  Also, reversing the order (first restart then
URL hit) also doesn't work.

-----Original Message-----
From: Anthony Messina [mailto:[e-mail]] 
Sent: Thursday, February 25, 2016 5:26 PM
To: [e-mail]
Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after
awhile

On Thursday, February 25, 2016 03:59:59 PM Simo Sorce wrote:
> On Thu, 2016-02-25 at 20:46 +0000, Wen Liang wrote:
> > Yes, restarting rpc.gssd (systemctl restart rpcgssd) also solves the 
> > Apache permission.
>
> Ok, his is good information,
> I wonder if there is a bug in rpc,gssd or in the gssproxy client.
>
> What version of gss-proxy are you running exactly ?
>
> Simo.

I can confirm this same issue, but I don't use gssproxy for the HTTP service
keytab, just the regular NFS client/server gssproxy.conf defaults to allow
Apache to serve KRB5/NFSv4 /home dirs. (Just didn't get a chance to report
it). -A

--
Anthony - https://messinet.com/ - https://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

[Robbie Harwood]

Wen Liang  writes:
> Here is a workaround that works for me.  Create a 5 minutes cron job that:
> hits an URL for the NFS mount and then restart gssproxy.
>
> Just restarting gssproxy by itself doesn't work.  The permission problem
> will eventually come back.  Also, reversing the order (first restart then
> URL hit) also doesn't work.

Hi,

I've backported a change to the locking that we made upstream.  Can you
test these RPMs and see if the problem changes?

https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf18/

Thanks,
--Robbie

[Wen Liang]

I will report back if the problem arises again.  So far so good...

-----Original Message-----
From: Robbie Harwood [mailto:[e-mail]] 
Sent: Thursday, March 3, 2016 1:17 PM
To: Wen Liang ; [e-mail]
Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after awhile

Wen Liang  writes:
> Here is a workaround that works for me.  Create a 5 minutes cron job that:
> hits an URL for the NFS mount and then restart gssproxy.
>
> Just restarting gssproxy by itself doesn't work.  The permission 
> problem will eventually come back.  Also, reversing the order (first 
> restart then URL hit) also doesn't work.

Hi,

I've backported a change to the locking that we made upstream.  Can you test these RPMs and see if the problem changes?

https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf18/

Thanks,
--Robbie

[Wen Liang]

Here is an update on the issue.  The test version worked better but the problem is still there.  Yesterday the problem came back, so I restarted gssproxy around 2pm.  Checked it around 7pm, no problem.  Checked again this morning around 9am, the problem came back.

-----Original Message-----
From: Robbie Harwood [mailto:[e-mail]] 
Sent: Thursday, March 3, 2016 1:17 PM
To: Wen Liang ; [e-mail]
Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after awhile

Wen Liang  writes:
> Here is a workaround that works for me.  Create a 5 minutes cron job that:
> hits an URL for the NFS mount and then restart gssproxy.
>
> Just restarting gssproxy by itself doesn't work.  The permission 
> problem will eventually come back.  Also, reversing the order (first 
> restart then URL hit) also doesn't work.

Hi,

I've backported a change to the locking that we made upstream.  Can you test these RPMs and see if the problem changes?

https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf18/

Thanks,
--Robbie

[Simo Sorce]

On Tue, 2016-03-08 at 11:18 -0500, Wen Liang wrote:
> Here is an update on the issue.  The test version worked better but
> the problem is still there.  Yesterday the problem came back, so I
> restarted gssproxy around 2pm.  Checked it around 7pm, no problem.
> Checked again this morning around 9am, the problem came back.

Thanks,
this means we are on the right track, but there is still something we
are missing.
Would you be able to attach strace to the (client) process and let it
log until you repro the error (it may create quite a big log, so it is
ok if you can't do that) and then share the log file with an indication
of the time around which you believe the issue happened (maybe also
attach the relative gss-proxy logs so we can determine the time).

Simo.
> -----Original Message-----
> From: Robbie Harwood [mailto:[e-mail]] 
> Sent: Thursday, March 3, 2016 1:17 PM
> To: Wen Liang ; [e-mail]
> Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after awhile
>
> Wen Liang  writes:
> > Here is a workaround that works for me.  Create a 5 minutes cron job that:
> > hits an URL for the NFS mount and then restart gssproxy.
> >
> > Just restarting gssproxy by itself doesn't work.  The permission 
> > problem will eventually come back.  Also, reversing the order (first 
> > restart then URL hit) also doesn't work.
>
> Hi,
>
> I've backported a change to the locking that we made upstream.  Can you test these RPMs and see if the problem changes?
>
> https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf18/
>
> Thanks,
> --Robbie
> _______________________________________________
> gss-proxy mailing list
> [e-mail]
> https://lists.fedorahosted.org/admin/lists/[e-mail]

[Wen Liang]

Okay, I put a strace on apache and its child processes.  Will report back when apache shows the permission problem again.

How do I send you the strace log output?

-----Original Message-----
From: Simo Sorce [mailto:[e-mail]] 
Sent: Tuesday, March 8, 2016 12:32 PM
To: [e-mail]
Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after awhile

On Tue, 2016-03-08 at 11:18 -0500, Wen Liang wrote:
> Here is an update on the issue.  The test version worked better but 
> the problem is still there.  Yesterday the problem came back, so I 
> restarted gssproxy around 2pm.  Checked it around 7pm, no problem.
> Checked again this morning around 9am, the problem came back.

Thanks,
this means we are on the right track, but there is still something we are missing.
Would you be able to attach strace to the (client) process and let it log until you repro the error (it may create quite a big log, so it is ok if you can't do that) and then share the log file with an indication of the time around which you believe the issue happened (maybe also attach the relative gss-proxy logs so we can determine the time).

Simo.
> -----Original Message-----
> From: Robbie Harwood [mailto:[e-mail]]
> Sent: Thursday, March 3, 2016 1:17 PM
> To: Wen Liang ; [e-mail]
> Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after 
> awhile
>
> Wen Liang  writes:
> > Here is a workaround that works for me.  Create a 5 minutes cron job that:
> > hits an URL for the NFS mount and then restart gssproxy.
> >
> > Just restarting gssproxy by itself doesn't work.  The permission 
> > problem will eventually come back.  Also, reversing the order (first 
> > restart then URL hit) also doesn't work.
>
> Hi,
>
> I've backported a change to the locking that we made upstream.  Can you test these RPMs and see if the problem changes?
>
> https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf1
> 8/
>
> Thanks,
> --Robbie
> _______________________________________________
> gss-proxy mailing list
> [e-mail]
> https://lists.fedorahosted.org/admin/lists/[e-mail]
> d.org

--
Simo Sorce * Red Hat, Inc * New York
_______________________________________________
gss-proxy mailing list
[e-mail]
https://lists.fedorahosted.org/admin/lists/[e-mail]

[Simo Sorce]

You can send it to me directly over email if it is not too big,
otherwise contact me and we'll find a way.

Simo.

On Tue, 2016-03-08 at 17:33 -0500, Wen Liang wrote:
> Okay, I put a strace on apache and its child processes.  Will report back when apache shows the permission problem again.
>
> How do I send you the strace log output?
>
> -----Original Message-----
> From: Simo Sorce [mailto:[e-mail]] 
> Sent: Tuesday, March 8, 2016 12:32 PM
> To: [e-mail]
> Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after awhile
>
> On Tue, 2016-03-08 at 11:18 -0500, Wen Liang wrote:
> > Here is an update on the issue.  The test version worked better but 
> > the problem is still there.  Yesterday the problem came back, so I 
> > restarted gssproxy around 2pm.  Checked it around 7pm, no problem.
> > Checked again this morning around 9am, the problem came back.
>
> Thanks,
> this means we are on the right track, but there is still something we are missing.
> Would you be able to attach strace to the (client) process and let it log until you repro the error (it may create quite a big log, so it is ok if you can't do that) and then share the log file with an indication of the time around which you believe the issue happened (maybe also attach the relative gss-proxy logs so we can determine the time).
>
> Simo.
> > -----Original Message-----
> > From: Robbie Harwood [mailto:[e-mail]]
> > Sent: Thursday, March 3, 2016 1:17 PM
> > To: Wen Liang ; [e-mail]
> > Subject: [gssproxy] Re: Gssproxy stops looking for Apache ccache after 
> > awhile
> >
> > Wen Liang  writes:
> > > Here is a workaround that works for me.  Create a 5 minutes cron job that:
> > > hits an URL for the NFS mount and then restart gssproxy.
> > >
> > > Just restarting gssproxy by itself doesn't work.  The permission 
> > > problem will eventually come back.  Also, reversing the order (first 
> > > restart then URL hit) also doesn't work.
> >
> > Hi,
> >
> > I've backported a change to the locking that we made upstream.  Can you test these RPMs and see if the problem changes?
> >
> > https://rharwood.fedorapeople.org/.94d9000d-f297-4602-bf3f-18c3b64dbf1
> > 8/
> >
> > Thanks,
> > --Robbie
> > _______________________________________________
> > gss-proxy mailing list
> > [e-mail]
> > https://lists.fedorahosted.org/admin/lists/[e-mail]
> > d.org
>
> --
> Simo Sorce * Red Hat, Inc * New York
> _______________________________________________
> gss-proxy mailing list
> [e-mail]
> https://lists.fedorahosted.org/admin/lists/[e-mail]
> _______________________________________________
> gss-proxy mailing list
> [e-mail]
> https://lists.fedorahosted.org/admin/lists/[e-mail]