Caching/performance issues with 1.5 vs 1.9

[Joshua C. Endries]

Hello,

I'm trying to get sssd going here to hook up with AD/LDAP for user and group lookup. I have it working, and it works great on RHEL5 (sssd v1.5.1). Running 'id' on myself takes 3s when in foreground mode, and 0.014s in service mode (service start...). Unfortunately, on RHEL6 (sssd v1.9.2), Running 'id' on myself takes 3-4min in foreground and 1min in service mode.  This is with the same sssd.conf file.

It looks like, when I look up my groups, it ends up looking up all the users in those groups, which 1.5 doesn't seem to do. We have a huge directory and caching all of this seems like a huge waste of resources... Is there a way to turn this off or modify this behavior? I tried reducing ldap_group_nesting_level but it didn't make a difference. Using ad instead of rfc2307bis didn't either. I didn't see anything else that looked like it would help...

Thanks,
Josh

[Dmitri Pal]

On 05/22/2013 04:26 PM, Joshua C. Endries wrote:
> Hello,
>
> I'm trying to get sssd going here to hook up with AD/LDAP for user and group lookup. I have it working, and it works great on RHEL5 (sssd v1.5.1). Running 'id' on myself takes 3s when in foreground mode, and 0.014s in service mode (service start...). Unfortunately, on RHEL6 (sssd v1.9.2), Running 'id' on myself takes 3-4min in foreground and 1min in service mode.  This is with the same sssd.conf file.
>
> It looks like, when I look up my groups, it ends up looking up all the users in those groups, which 1.5 doesn't seem to do. We have a huge directory and caching all of this seems like a huge waste of resources... Is there a way to turn this off or modify this behavior? I tried reducing ldap_group_nesting_level but it didn't make a difference. Using ad instead of rfc2307bis didn't either. I didn't see anything else that looked like it would help...
>
> Thanks,
> Josh
> _______________________________________________
> sssd-users mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-users

Can you post the sssd.conf file?

[Jakub Hrozek]

On Wed, May 22, 2013 at 08:26:25PM +0000, Joshua C. Endries wrote:
> Hello,
>
> I'm trying to get sssd going here to hook up with AD/LDAP for user and group lookup. I have it working, and it works great on RHEL5 (sssd v1.5.1). Running 'id' on myself takes 3s when in foreground mode, and 0.014s in service mode (service start...). Unfortunately, on RHEL6 (sssd v1.9.2), Running 'id' on myself takes 3-4min in foreground and 1min in service mode.  This is with the same sssd.conf file.
>
> It looks like, when I look up my groups, it ends up looking up all the users in those groups, which 1.5 doesn't seem to do. We have a huge directory and caching all of this seems like a huge waste of resources... Is there a way to turn this off or modify this behavior? I tried reducing ldap_group_nesting_level but it didn't make a difference. Using ad instead of rfc2307bis didn't either. I didn't see anything else that looked like it would help...
>
> Thanks,
> Josh

Hi Joshua,

it seems you are running into https://fedorahosted.org/sssd/ticket/1823

Before we have a more systematic fix we'll be adding a new option to
disable the range retrieval altogether when that option is set. That
should bring the same performance as you had with 1.5

[Jakub Hrozek]

On Thu, May 23, 2013 at 10:36:21AM +0200, Jakub Hrozek wrote:
> On Wed, May 22, 2013 at 08:26:25PM +0000, Joshua C. Endries wrote:
> > Hello,
> >
> > I'm trying to get sssd going here to hook up with AD/LDAP for user and group lookup. I have it working, and it works great on RHEL5 (sssd v1.5.1). Running 'id' on myself takes 3s when in foreground mode, and 0.014s in service mode (service start...). Unfortunately, on RHEL6 (sssd v1.9.2), Running 'id' on myself takes 3-4min in foreground and 1min in service mode.  This is with the same sssd.conf file.
> >
> > It looks like, when I look up my groups, it ends up looking up all the users in those groups, which 1.5 doesn't seem to do. We have a huge directory and caching all of this seems like a huge waste of resources... Is there a way to turn this off or modify this behavior? I tried reducing ldap_group_nesting_level but it didn't make a difference. Using ad instead of rfc2307bis didn't either. I didn't see anything else that looked like it would help...
> >
> > Thanks,
> > Josh
>
> Hi Joshua,
>
> it seems you are running into https://fedorahosted.org/sssd/ticket/1823
>
> Before we have a more systematic fix we'll be adding a new option to
> disable the range retrieval altogether when that option is set. That
> should bring the same performance as you had with 1.5

I forgot to add -- we already have a patch ready. Would you be
interested in testing it out?

[Josh Endries]

I would definitely be interested in testing the changes out.

I don't think I am running into that ticket exactly; I'm not in one group with that many users that I'm aware of. However, my own account is in over twenty groups, some of which are "all employees" and "all students", so it's a large result set. Ultimately it just means lots and lots of extra look-ups when I just want a list of GIDs/names.

Here is my config file. This is mostly from trial and error, Google and man, so it's probably not perfect (but it works):

# grep -vE '^(#|$)' sssd.conf
[sssd]
config_file_version = 2
domains = CUAD
services = nss, pam
[nss]
debug_level = 0
filter_users = root
filter_groups = root
[domain/CUAD]
auth_provider = krb5
enumerate = false
id_provider = ldap
krb5_realm = ...
krb5_server = ...
ldap_default_bind_dn = ...
ldap_default_authtok_type = password
ldap_default_authtok = ...
ldap_disable_referrals = true
ldap_group_object_class = group
ldap_id_use_start_tls = true
ldap_schema = rfc2307bis
ldap_search_base = ...
ldap_tls_reqcert = allow
ldap_uri = ldaps://...
ldap_user_fullname = displayName
ldap_user_gecos = displayName
ldap_user_home_directory = unixHomeDirectory
ldap_user_name = cn
ldap_user_object_class = user

Thanks,
Josh

----- Original Message -----
From: "Jakub Hrozek" 
To: [e-mail]
Sent: Thursday, May 23, 2013 4:44:13 AM
Subject: Re: [SSSD-users] Caching/performance issues with 1.5 vs 1.9

On Thu, May 23, 2013 at 10:36:21AM +0200, Jakub Hrozek wrote:
> On Wed, May 22, 2013 at 08:26:25PM +0000, Joshua C. Endries wrote:
> > Hello,
> >
> > I'm trying to get sssd going here to hook up with AD/LDAP for user and group lookup. I have it working, and it works great on RHEL5 (sssd v1.5.1). Running 'id' on myself takes 3s when in foreground mode, and 0.014s in service mode (service start...). Unfortunately, on RHEL6 (sssd v1.9.2), Running 'id' on myself takes 3-4min in foreground and 1min in service mode.  This is with the same sssd.conf file.
> >
> > It looks like, when I look up my groups, it ends up looking up all the users in those groups, which 1.5 doesn't seem to do. We have a huge directory and caching all of this seems like a huge waste of resources... Is there a way to turn this off or modify this behavior? I tried reducing ldap_group_nesting_level but it didn't make a difference. Using ad instead of rfc2307bis didn't either. I didn't see anything else that looked like it would help...
> >
> > Thanks,
> > Josh
>
> Hi Joshua,
>
> it seems you are running into https://fedorahosted.org/sssd/ticket/1823
>
> Before we have a more systematic fix we'll be adding a new option to
> disable the range retrieval altogether when that option is set. That
> should bring the same performance as you had with 1.5

I forgot to add -- we already have a patch ready. Would you be
interested in testing it out?
_______________________________________________
sssd-users mailing list
[e-mail]
https://lists.fedorahosted.org/mailman/listinfo/sssd-users

[Jakub Hrozek]

On Thu, May 23, 2013 at 07:59:14AM -0400, Josh Endries wrote:
> I would definitely be interested in testing the changes out.

Great, I build the latest 6.4 packages along with the new option to
disable range retrievals:
http://jhrozek.fedorapeople.org/sssd-range-retrieval/

To disable the range retrieval functionality (and get the same behaviour
as in 6.3), put the following directive into your sssd.conf into the
domain section:

ldap_disable_range_retrieval = True

and then restart the SSSD. Large groups (>1500 members) should then
appear as empty, while small groups should appear as they used to.
> I don't think I am running into that ticket exactly; I'm not in one group
> with that many users that I'm aware of. However, my own account is in over
> twenty groups, some of which are "all employees" and "all students", so
> it's a large result set. Ultimately it just means lots and lots of extra
> look-ups when I just want a list of GIDs/names.

I see, then it might be a completely different issue. I would advise to
test the build first and if it doesn't help, then we'd take a look at
the debug logs.
> Here is my config file. This is mostly from trial and error, Google and man, so it's probably not perfect (but it works):

The config file looks good to me, in general I would just recommend
using GSSAPI over password binds:
https://fedorahosted.org/sssd/wiki/Configuring%20sssd%20to%20authenticate%20...

The most important part for performance when it comes to AD client is
disabling referrals (which you already do).