SSSD netgroup issue

[Francisco Javier Marín Murillo]

Hello,

I have this nsswitch configuration:

passwd: compat
passwd_compat:  sss
#shadow: files ldap
#group:  files ldap

#passwd:     files sss
#passwd:            compat sss
shadow:     files sss
group:      files sss

hosts:      files dns
bootparams: files
ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files
netgroup:   sss
publickey:  nisplus
automount:  files ldap
aliases:    files
sudoers:    files ldap

Currently I am having problems with sssd handling netgroup changes.

when I add or remove a user from a netgroup , those changes are not replicated automatically to clients . Should be something wrong with my sssd.conf configuration ?

If you can give me a hand I would really appreciate it.

Thanks,

Francisco Marin

[Jan Zelený]

> Hello,
>
> I have this nsswitch configuration:
>
> passwd: compat
> passwd_compat:  sss
> #shadow: files ldap
> #group:  files ldap
>
> #passwd:     files sss
> #passwd:            compat sss
> shadow:     files sss
> group:      files sss
>
> hosts:      files dns
> bootparams: files
> ethers:     files
> netmasks:   files
> networks:   files
> protocols:  files
> rpc:        files
> services:   files
> netgroup:   sss
> publickey:  nisplus
> automount:  files ldap
> aliases:    files
> sudoers:    files ldap
>
> Currently I am having problems with sssd handling netgroup changes.
>
> when I add or remove a user from a netgroup , those changes are not
> replicated automatically to clients . Should be something wrong with my
> sssd.conf configuration ?
>
> If you can give me a hand I would really appreciate it.
>
> Thanks,
>
> Francisco Marin

Hi,
my first guess would be cache expiration. Did you try running sss_cache -N on 
the client before trying if the propagation worked?

Jan

[Stephen Gallagher]

On Thu, 2011-09-22 at 13:50 +0200, Jan Zelený wrote:
> > Hello,
> >
> > I have this nsswitch configuration:
> >
> > passwd: compat
> > passwd_compat:  sss
> > #shadow: files ldap
> > #group:  files ldap
> >
> > #passwd:     files sss
> > #passwd:            compat sss
> > shadow:     files sss
> > group:      files sss
> >
> > hosts:      files dns
> > bootparams: files
> > ethers:     files
> > netmasks:   files
> > networks:   files
> > protocols:  files
> > rpc:        files
> > services:   files
> > netgroup:   sss
> > publickey:  nisplus
> > automount:  files ldap
> > aliases:    files
> > sudoers:    files ldap
> >
> > Currently I am having problems with sssd handling netgroup changes.
> >
> > when I add or remove a user from a netgroup , those changes are not
> > replicated automatically to clients . Should be something wrong with my
> > sssd.conf configuration ?
> >
> > If you can give me a hand I would really appreciate it.
> >
> > Thanks,
> >
> > Francisco Marin
>
> Hi,
> my first guess would be cache expiration. Did you try running sss_cache -N on 
> the client before trying if the propagation worked?

The issue here is almost certainly cache expiration. Once you have
requested a netgroup, it will be available for entry_cache_timeout
seconds (defaults to 90 minutes). So changing the value on the server
won't have an effect on the client until up to 90 minutes later.

You can change this timeout by setting the following in the
[domain/DOMAINNAME] section of sssd.conf:
entry_cache_timeout = 90
(to set it to 90 seconds)

You can also purge your cache immediately by stopping SSSD and removing
the file /var/lib/sss/db/cache_DOMAINNAME.ldb.

In SSSD 1.6.0 and later, you can also use the sss_cache command like Jan
described above to immediately expire all cache entries (without
deleting the cache).

[Francisco Javier Marín Murillo]

Thank you very much Jan . I could not find sss_cache command, is there a package that I am missing ? . But propagation should not happen automatically ? or should I run sss_cache -N always ?.

is not a parameter on my sssd.conf file which would let me propagate the changes automatically ?

Thank you again. Regards,

Francisco Marin
> From: [e-mail]
> To: [e-mail]
> Subject: Re: [SSSD] SSSD netgroup issue
> Date: Thu, 22 Sep 2011 13:50:01 +0200
> CC: [e-mail]
> > Hello,
> >
> > I have this nsswitch configuration:
> >
> > passwd: compat
> > passwd_compat: sss
> > #shadow: files ldap
> > #group: files ldap
> >
> > #passwd: files sss
> > #passwd: compat sss
> > shadow: files sss
> > group: files sss
> >
> > hosts: files dns
> > bootparams: files
> > ethers: files
> > netmasks: files
> > networks: files
> > protocols: files
> > rpc: files
> > services: files
> > netgroup: sss
> > publickey: nisplus
> > automount: files ldap
> > aliases: files
> > sudoers: files ldap
> >
> > Currently I am having problems with sssd handling netgroup changes.
> >
> > when I add or remove a user from a netgroup , those changes are not
> > replicated automatically to clients . Should be something wrong with my
> > sssd.conf configuration ?
> >
> > If you can give me a hand I would really appreciate it.
> >
> > Thanks,
> >
> > Francisco Marin
>
> Hi,
> my first guess would be cache expiration. Did you try running sss_cache -N on 
> the client before trying if the propagation worked?
>
> Jan

[Jan Zelený]

> Thank you very much Jan . I could not find sss_cache command, is there a
> package that I am missing ? . But propagation should not happen
> automatically ? or should I run sss_cache -N always ?.
>
> is not a parameter on my sssd.conf file which would let me propagate the
> changes automatically ?
>
> Thank you again. Regards,

As Stephen pointed out, the sss_cache command is available since 1.6.0. For 
more specifics of cache timeout, please see his email. Answers to all your 
questions are there.
> Francisco Marin
> > From: [e-mail]
> > To: [e-mail]
> > Subject: Re: [SSSD] SSSD netgroup issue
> > Date: Thu, 22 Sep 2011 13:50:01 +0200
> > CC: [e-mail]
> > > Hello,
> > >
> > > I have this nsswitch configuration:
> > >
> > > passwd: compat
> > > passwd_compat: sss
> > > #shadow: files ldap
> > > #group: files ldap
> > >
> > > #passwd: files sss
> > > #passwd: compat sss
> > > shadow: files sss
> > > group: files sss
> > >
> > > hosts: files dns
> > > bootparams: files
> > > ethers: files
> > > netmasks: files
> > > networks: files
> > > protocols: files
> > > rpc: files
> > > services: files
> > > netgroup: sss
> > > publickey: nisplus
> > > automount: files ldap
> > > aliases: files
> > > sudoers: files ldap
> > >
> > > Currently I am having problems with sssd handling netgroup changes.
> > >
> > > when I add or remove a user from a netgroup , those changes are not
> > > replicated automatically to clients . Should be something wrong with my
> > > sssd.conf configuration ?
> > >
> > > If you can give me a hand I would really appreciate it.
> > >
> > > Thanks,
> > >
> > > Francisco Marin
> >
> > Hi,
> > my first guess would be cache expiration. Did you try running sss_cache
> > -N on the client before trying if the propagation worked?
> >
> > Jan

[Francisco Javier Marín Murillo]

Hello,

I do not see Stephen's answers that you mentioned about. Can he send email again ?

Thank you very much. Regards,

Francisco Marin
> From: [e-mail]
> To: [e-mail]
> Subject: Re: [SSSD] SSSD netgroup issue
> Date: Fri, 23 Sep 2011 08:49:59 +0200
> CC: [e-mail]
> > Thank you very much Jan . I could not find sss_cache command, is there a
> > package that I am missing ? . But propagation should not happen
> > automatically ? or should I run sss_cache -N always ?.
> >
> > is not a parameter on my sssd.conf file which would let me propagate the
> > changes automatically ?
> >
> > Thank you again. Regards,
>
> As Stephen pointed out, the sss_cache command is available since 1.6.0. For 
> more specifics of cache timeout, please see his email. Answers to all your 
> questions are there.
> > Francisco Marin
> > > From: [e-mail]
> > > To: [e-mail]
> > > Subject: Re: [SSSD] SSSD netgroup issue
> > > Date: Thu, 22 Sep 2011 13:50:01 +0200
> > > CC: [e-mail]
> > > > Hello,
> > > >
> > > > I have this nsswitch configuration:
> > > >
> > > > passwd: compat
> > > > passwd_compat: sss
> > > > #shadow: files ldap
> > > > #group: files ldap
> > > >
> > > > #passwd: files sss
> > > > #passwd: compat sss
> > > > shadow: files sss
> > > > group: files sss
> > > >
> > > > hosts: files dns
> > > > bootparams: files
> > > > ethers: files
> > > > netmasks: files
> > > > networks: files
> > > > protocols: files
> > > > rpc: files
> > > > services: files
> > > > netgroup: sss
> > > > publickey: nisplus
> > > > automount: files ldap
> > > > aliases: files
> > > > sudoers: files ldap
> > > >
> > > > Currently I am having problems with sssd handling netgroup changes.
> > > >
> > > > when I add or remove a user from a netgroup , those changes are not
> > > > replicated automatically to clients . Should be something wrong with my
> > > > sssd.conf configuration ?
> > > >
> > > > If you can give me a hand I would really appreciate it.
> > > >
> > > > Thanks,
> > > >
> > > > Francisco Marin
> > >
> > > Hi,
> > > my first guess would be cache expiration. Did you try running sss_cache
> > > -N on the client before trying if the propagation worked?
> > >
> > > Jan
>
> -- 
> Thank you
> Jan Zeleny
>
> Red Hat Software Engineer
> Brno, Czech Republic

[Stephen Gallagher]

On Fri, 2011-09-23 at 12:42 -0600, Francisco Javier Marín Murillo wrote:
> Hello,
>
> I do not see Stephen's answers that you mentioned about. Can he send
> email again ?

Please subscribe to the sssd-devel mailing list to see all replies. We
usually don't reply directly to the original email address because the
responses may be of interest to others on the list as well. You can
subscribe at https://fedorahosted.org/mailman/listinfo/sssd-devel

From my earlier mail to this list:

The issue here is almost certainly cache expiration. Once you have
requested a netgroup, it will be available for entry_cache_timeout
seconds (defaults to 90 minutes). So changing the value on the server
won't have an effect on the client until up to 90 minutes later.

You can change this timeout by setting the following in the
[domain/DOMAINNAME] section of sssd.conf:
entry_cache_timeout = 90
(to set it to 90 seconds)

You can also purge your cache immediately by stopping SSSD and removing
the file /var/lib/sss/db/cache_DOMAINNAME.ldb.

In SSSD 1.6.0 and later, you can also use the sss_cache command like Jan
described above to immediately expire all cache entries (without
deleting the cache).

As an addendum, I strongly recommend against setting the cache timeout
shorter than about five or ten seconds, as you don't want to hammer your
LDAP server if someone does an 'ls -l' on a directory with a lot of
files.

[Francisco Javier Marín Murillo]

Just to let you know the only way how I have been able to expire netgroup cache is when I delete db cache and restart sssd. But that does not work for us because we want sssd to expire cache automatically with no manual intervention.

If you can shed on this I would really appreciate it.

Thank you very much. Regards,

Francisco Marin
> From: [e-mail]
> To: [e-mail]
> Subject: Re: [SSSD] SSSD netgroup issue
> Date: Fri, 23 Sep 2011 08:49:59 +0200
> CC: [e-mail]
> > Thank you very much Jan . I could not find sss_cache command, is there a
> > package that I am missing ? . But propagation should not happen
> > automatically ? or should I run sss_cache -N always ?.
> >
> > is not a parameter on my sssd.conf file which would let me propagate the
> > changes automatically ?
> >
> > Thank you again. Regards,
>
> As Stephen pointed out, the sss_cache command is available since 1.6.0. For 
> more specifics of cache timeout, please see his email. Answers to all your 
> questions are there.
> > Francisco Marin
> > > From: [e-mail]
> > > To: [e-mail]
> > > Subject: Re: [SSSD] SSSD netgroup issue
> > > Date: Thu, 22 Sep 2011 13:50:01 +0200
> > > CC: [e-mail]
> > > > Hello,
> > > >
> > > > I have this nsswitch configuration:
> > > >
> > > > passwd: compat
> > > > passwd_compat: sss
> > > > #shadow: files ldap
> > > > #group: files ldap
> > > >
> > > > #passwd: files sss
> > > > #passwd: compat sss
> > > > shadow: files sss
> > > > group: files sss
> > > >
> > > > hosts: files dns
> > > > bootparams: files
> > > > ethers: files
> > > > netmasks: files
> > > > networks: files
> > > > protocols: files
> > > > rpc: files
> > > > services: files
> > > > netgroup: sss
> > > > publickey: nisplus
> > > > automount: files ldap
> > > > aliases: files
> > > > sudoers: files ldap
> > > >
> > > > Currently I am having problems with sssd handling netgroup changes.
> > > >
> > > > when I add or remove a user from a netgroup , those changes are not
> > > > replicated automatically to clients . Should be something wrong with my
> > > > sssd.conf configuration ?
> > > >
> > > > If you can give me a hand I would really appreciate it.
> > > >
> > > > Thanks,
> > > >
> > > > Francisco Marin
> > >
> > > Hi,
> > > my first guess would be cache expiration. Did you try running sss_cache
> > > -N on the client before trying if the propagation worked?
> > >
> > > Jan
>
> -- 
> Thank you
> Jan Zeleny
>
> Red Hat Software Engineer
> Brno, Czech Republic

[Stephen Gallagher]

On Fri, 2011-09-23 at 13:00 -0600, Francisco Javier Marín Murillo wrote:
> Just to let you know the only way how I have been able to expire
> netgroup cache is when I delete db cache and restart sssd. But that
> does not work for us because we want sssd to expire cache
> automatically with no manual intervention.

As I wrote in my other email, there will always be a lag, based on the
entry_cache_timeout value. This is to reduce the load on your LDAP
server, under the reasonable expectation that entries in LDAP are
"write-rarely, read often". In the majority of cases, you don't want to
waste time and CPU on constantly going out the LDAP server.

For the reverse, there's no way for the LDAP server to "push" updates to
the clients. LDAP doesn't work that way. All data requests have to
originate with the clients. So there's no way to achieve an
instantaneous update when something changes.

[Francisco Javier Marín Murillo]

Thank you Stephen. But I set  entry_cache_timeout  to 90 seconds.  The issue is that even setting it to 90 seconds or 5 seconds it never times out(even after 90 seconds or 5 seconds is expired).  It never ever expires.  I have checked the client the next day and the entry is still in the database   That is the issue.  What can be causing this ? is it something wrong with the sssd service that does not read correctly the sssd.conf configurations ?
> Subject: Re: [SSSD] SSSD netgroup issue
> From: [e-mail]
> To: [e-mail]
> CC: [e-mail]
> Date: Fri, 23 Sep 2011 15:09:40 -0400
>
> On Fri, 2011-09-23 at 13:00 -0600, Francisco Javier Marín Murillo wrote:
> > Just to let you know the only way how I have been able to expire
> > netgroup cache is when I delete db cache and restart sssd. But that
> > does not work for us because we want sssd to expire cache
> > automatically with no manual intervention.
>
> As I wrote in my other email, there will always be a lag, based on the
> entry_cache_timeout value. This is to reduce the load on your LDAP
> server, under the reasonable expectation that entries in LDAP are
> "write-rarely, read often". In the majority of cases, you don't want to
> waste time and CPU on constantly going out the LDAP server.
>
> For the reverse, there's no way for the LDAP server to "push" updates to
> the clients. LDAP doesn't work that way. All data requests have to
> originate with the clients. So there's no way to achieve an
> instantaneous update when something changes.

[Dmitri Pal]

On 09/23/2011 05:55 PM, Francisco Javier Marín Murillo wrote:
> Thank you Stephen. But I set  entry_cache_timeout  to 90 seconds.  The
> issue is that even setting it to 90 seconds or 5 seconds it never
> times out(even after 90 seconds or 5 seconds is expired).  It never
> ever expires.  I have checked the client the next day and the entry is
> still in the database   That is the issue.  What can be causing this ?
> is it something wrong with the sssd service that does not read
> correctly the sssd.conf configurations ?

Might be a bug...
Stephen it is worth inspecting the code where we check the expiration
timeout may be we do something wrong there.
> > Subject: Re: [SSSD] SSSD netgroup issue
> > From: [e-mail]
> > To: [e-mail]
> > CC: [e-mail] 
> > Date: Fri, 23 Sep 2011 15:09:40 -0400
> >
> > On Fri, 2011-09-23 at 13:00 -0600, Francisco Javier Marín Murillo wrote:
> > > Just to let you know the only way how I have been able to expire
> > > netgroup cache is when I delete db cache and restart sssd. But that
> > > does not work for us because we want sssd to expire cache
> > > automatically with no manual intervention.
> >
> > As I wrote in my other email, there will always be a lag, based on the
> > entry_cache_timeout value. This is to reduce the load on your LDAP
> > server, under the reasonable expectation that entries in LDAP are
> > "write-rarely, read often". In the majority of cases, you don't want to
> > waste time and CPU on constantly going out the LDAP server.
> >
> > For the reverse, there's no way for the LDAP server to "push" updates to
> > the clients. LDAP doesn't work that way. All data requests have to
> > originate with the clients. So there's no way to achieve an
> > instantaneous update when something changes.
>
> sssd-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/sssd-devel

[Stephen Gallagher]

On Sep 23, 2011, at 5:55 PM, Francisco Javier Marín Murillo    wrote:
> Thank you Stephen. But I set  entry_cache_timeout  to 90 seconds.  The issue is that even setting it to 90 seconds or 5 seconds it never times out(even after 90 seconds or 5 seconds is expired).  It never ever expires.  I have checked the client the next day and the entry is still in the database   That is the issue.  What can be causing this ? is it something wrong with the sssd service that does not read correctly the sssd.conf configurations ?

We never remove the entry from the cache. We will update it the next time it is requested after the expiration is reached. We do this so that offline operation can continue.

Offline operation is defined as any time that the LDAP server cannot be reached.
> > Subject: Re: [SSSD] SSSD netgroup issue
> > From: [e-mail]
> > To: [e-mail]
> > CC: [e-mail]
> > Date: Fri, 23 Sep 2011 15:09:40 -0400
> >
> > On Fri, 2011-09-23 at 13:00 -0600, Francisco Javier Marín Murillo wrote:
> > > Just to let you know the only way how I have been able to expire
> > > netgroup cache is when I delete db cache and restart sssd. But that
> > > does not work for us because we want sssd to expire cache
> > > automatically with no manual intervention.
> >
> > As I wrote in my other email, there will always be a lag, based on the
> > entry_cache_timeout value. This is to reduce the load on your LDAP
> > server, under the reasonable expectation that entries in LDAP are
> > "write-rarely, read often". In the majority of cases, you don't want to
> > waste time and CPU on constantly going out the LDAP server.
> >
> > For the reverse, there's no way for the LDAP server to "push" updates to
> > the clients. LDAP doesn't work that way. All data requests have to
> > originate with the clients. So there's no way to achieve an
> > instantaneous update when something changes.

[Francisco Javier Marín Murillo]

Ok,  But I set  entry_cache_timeout  to 90 seconds.  The issue is that even setting it to 90 seconds or 5 seconds it never times out(even after 90 seconds or 5 seconds is expired).  It never ever expires.  I have checked the client the next day and the entry is still in the database   That is the issue.  What can be causing this ? is it something wrong with the sssd service that does not read correctly the sssd.conf configurations ?

Subject: Re: [SSSD] SSSD netgroup issue
From: [e-mail]
Date: Fri, 23 Sep 2011 18:09:28 -0400
To: [e-mail]
CC: [e-mail]

On Sep 23, 2011, at 5:55 PM, Francisco Javier Marín Murillo    wrote:

Thank you Stephen. But I set  entry_cache_timeout  to 90 seconds.  The issue is that even setting it to 90 seconds or 5 seconds it never times out(even after 90 seconds or 5 seconds is expired).  It never ever expires.  I have checked the client the next day and the entry is still in the database   That is the issue.  What can be causing this ? is it something wrong with the sssd service that does not read correctly the sssd.conf configurations ?

We never remove the entry from the cache. We will update it the next time it is requested after the expiration is reached. We do this so that offline operation can continue.

Offline operation is defined as any time that the LDAP server cannot be reached.
> Subject: Re: [SSSD] SSSD netgroup issue
> From: [e-mail]
> To: [e-mail]
> CC: [e-mail]
> Date: Fri, 23 Sep 2011 15:09:40 -0400
>
> On Fri, 2011-09-23 at 13:00 -0600, Francisco Javier Marín Murillo wrote:
> > Just to let you know the only way how I have been able to expire
> > netgroup cache is when I delete db cache and restart sssd. But that
> > does not work for us because we want sssd to expire cache
> > automatically with no manual intervention.
>
> As I wrote in my other email, there will always be a lag, based on the
> entry_cache_timeout value. This is to reduce the load on your LDAP
> server, under the reasonable expectation that entries in LDAP are
> "write-rarely, read often". In the majority of cases, you don't want to
> waste time and CPU on constantly going out the LDAP server.
>
> For the reverse, there's no way for the LDAP server to "push" updates to
> the clients. LDAP doesn't work that way. All data requests have to
> originate with the clients. So there's no way to achieve an
> instantaneous update when something changes.

[Simo Sorce]

On Fri, 2011-09-23 at 17:56 -0600, Francisco Javier Marín Murillo wrote:
> Ok,  But I set  entry_cache_timeout  to 90 seconds.  The issue is that
> even setting it to 90 seconds or 5 seconds it never times out(even
> after 90 seconds or 5 seconds is expired).  It never ever expires.  I
> have checked the client the next day and the entry is still in the
> database   That is the issue.  What can be causing this ? is
> it something wrong with the sssd service that does not read correctly
> the sssd.conf configurations ?

It would be an issue if you were still getting the old info back on
query when online.

We do not remove the information form the database, we simply mark it
expired, so if you are online and you query the netgroups database sssd
should refresh the information before returning you any result.

If this is failing (ie sssd is returning outdated information while
online and after the cache has expired) please open a ticket as in this
it is a bug.

Simo.