[PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Michal Židek]

Hi,

this patch solves the ticket:
https://fedorahosted.org/sssd/ticket/2361

From the discussion on bugzilla and the ticket comments, it
looks like Jakub and Lukas were for changes in documentation
only (+ maybe warning/error if user misconfigured SSSD).

I do not want look like I ignore the opinions of others
but I think the best way is to simply change the defaults
to the same as expected by AD backend. While I do agree
that users should set their id_provider option to AD
if they use AD backend, this option is more or less
AD specific, so I think it would be better to have
the same defaults with id_provider ldap. This
would also solve the small regressions that are
mentioned in the bugzilla (by users who do use
id_provider=AD and id_mapping=true).

OTOH if this approach is disliked by other developers
I have no big problems with the man page + warning/error
solution.

Simple patch is attached.

Thanks,
Michal

[Pavel Reichl]

On 09/29/2014 05:58 PM, Michal Židek wrote:
> Hi,
>
> this patch solves the ticket:
> https://fedorahosted.org/sssd/ticket/2361
>
> From the discussion on bugzilla and the ticket comments, it
> looks like Jakub and Lukas were for changes in documentation
> only (+ maybe warning/error if user misconfigured SSSD).
>
> I do not want look like I ignore the opinions of others
> but I think the best way is to simply change the defaults
> to the same as expected by AD backend. While I do agree
> that users should set their id_provider option to AD
> if they use AD backend, this option is more or less
> AD specific, so I think it would be better to have
> the same defaults with id_provider ldap. This
> would also solve the small regressions that are
> mentioned in the bugzilla (by users who do use
> id_provider=AD and id_mapping=true).
>
> OTOH if this approach is disliked by other developers
> I have no big problems with the man page + warning/error
> solution.
>
> Simple patch is attached.
>
> Thanks,
> Michal

ACK,

this patch is also useful for https://fedorahosted.org/sssd/ticket/2117
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Jakub Hrozek]

On Mon, Oct 13, 2014 at 01:53:29PM +0200, Pavel Reichl wrote:
> On 09/29/2014 05:58 PM, Michal Židek wrote:
> > Hi,
> >
> > this patch solves the ticket:
> > https://fedorahosted.org/sssd/ticket/2361
> >
> > From the discussion on bugzilla and the ticket comments, it
> > looks like Jakub and Lukas were for changes in documentation
> > only (+ maybe warning/error if user misconfigured SSSD).
> >
> > I do not want look like I ignore the opinions of others
> > but I think the best way is to simply change the defaults
> > to the same as expected by AD backend. While I do agree
> > that users should set their id_provider option to AD
> > if they use AD backend, this option is more or less
> > AD specific, so I think it would be better to have
> > the same defaults with id_provider ldap. This
> > would also solve the small regressions that are
> > mentioned in the bugzilla (by users who do use
> > id_provider=AD and id_mapping=true).
> >
> > OTOH if this approach is disliked by other developers
> > I have no big problems with the man page + warning/error
> > solution.
> >
> > Simple patch is attached.
> >
> > Thanks,
> > Michal
>
> ACK,
>
> this patch is also useful for https://fedorahosted.org/sssd/ticket/2117

Yes, after some discussion I don't see a disadvantage either. Users with
id_provider=ldap connected to AD will benefit from this change and users
with id_provider=ldap connected to IPA will not be hit, since they would
run with non-defaults already.

Given the NULL default has created some confusion and broke
configurations that ran fine (even if by accident a bit), I agree with
the change, too.

Pushed to master as: f834f712548db811695ea0fd6d6b31d3bd03e2a3

[Jakub Hrozek]

On Thu, Oct 16, 2014 at 06:19:34PM +0200, Jakub Hrozek wrote:
> On Mon, Oct 13, 2014 at 01:53:29PM +0200, Pavel Reichl wrote:
> > On 09/29/2014 05:58 PM, Michal Židek wrote:
> > > Hi,
> > >
> > > this patch solves the ticket:
> > > https://fedorahosted.org/sssd/ticket/2361
> > >
> > > From the discussion on bugzilla and the ticket comments, it
> > > looks like Jakub and Lukas were for changes in documentation
> > > only (+ maybe warning/error if user misconfigured SSSD).
> > >
> > > I do not want look like I ignore the opinions of others
> > > but I think the best way is to simply change the defaults
> > > to the same as expected by AD backend. While I do agree
> > > that users should set their id_provider option to AD
> > > if they use AD backend, this option is more or less
> > > AD specific, so I think it would be better to have
> > > the same defaults with id_provider ldap. This
> > > would also solve the small regressions that are
> > > mentioned in the bugzilla (by users who do use
> > > id_provider=AD and id_mapping=true).
> > >
> > > OTOH if this approach is disliked by other developers
> > > I have no big problems with the man page + warning/error
> > > solution.
> > >
> > > Simple patch is attached.
> > >
> > > Thanks,
> > > Michal
> >
> > ACK,
> >
> > this patch is also useful for https://fedorahosted.org/sssd/ticket/2117
>
> Yes, after some discussion I don't see a disadvantage either. Users with
> id_provider=ldap connected to AD will benefit from this change and users
> with id_provider=ldap connected to IPA will not be hit, since they would
> run with non-defaults already.
>
> Given the NULL default has created some confusion and broke
> configurations that ran fine (even if by accident a bit), I agree with
> the change, too.
>
> Pushed to master as: f834f712548db811695ea0fd6d6b31d3bd03e2a3

Also pushed to sssd-1-11: 29e5b5d17d9700022958bf1f59bb861cdf68bb57

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Lukas Slebodnik]

On (29/09/14 17:58), Michal Židek wrote:
> Hi,
>
> this patch solves the ticket:
> https://fedorahosted.org/sssd/ticket/2361
>
> From the discussion on bugzilla and the ticket comments, it
> looks like Jakub and Lukas were for changes in documentation
> only (+ maybe warning/error if user misconfigured SSSD).
>
> I do not want look like I ignore the opinions of others
> but I think the best way is to simply change the defaults
> to the same as expected by AD backend. While I do agree
> that users should set their id_provider option to AD
> if they use AD backend, this option is more or less
> AD specific, so I think it would be better to have
> the same defaults with id_provider ldap. This
> would also solve the small regressions that are
> mentioned in the bugzilla (by users who do use
> id_provider=AD and id_mapping=true).
>
> OTOH if this approach is disliked by other developers
> I have no big problems with the man page + warning/error
> solution.
>
> Simple patch is attached.
>
> Thanks,
> Michal
> From 2d0f2f3c62eefe4f55cb21b91538bad8b68f6df5 Mon Sep 17 00:00:00 2001
> From: Michal Zidek 
> Date: Mon, 29 Sep 2014 17:17:59 +0200
> Subject: [PATCH] LDAP: Change defaults for ldap_user/group_objectsid
>
> Fixes:
> https://fedorahosted.org/sssd/ticket/2361
>
> src/man/sssd-ldap.5.xml        | 4 ++--
> src/providers/ldap/ldap_opts.h | 8 ++++----
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/src/man/sssd-ldap.5.xml b/src/man/sssd-ldap.5.xml
> index 03ea794..a21ffc1 100644
> --- a/src/man/sssd-ldap.5.xml
> +++ b/src/man/sssd-ldap.5.xml
> @@ -346,7 +346,7 @@
>                             necessary for ActiveDirectory servers.
>                         </para>
>                         <para>
>                        Default: objectSid for ActiveDirectory, not set
>                        Default: ipaNTSecurityIdentifier for IPA, objectSID
>                       for other servers.
>                   </para>
>               </listitem>
>
> @@ -854,7 +854,7 @@
>                             necessary for ActiveDirectory servers.
>                         </para>
>                         <para>
>                        Default: objectSid for ActiveDirectory, not set
>                        Default: ipaNTSecurityIdentifier for IPA, objectSID
>                       for other servers.
>                   </para>
>               </listitem>
>
> diff --git a/src/providers/ldap/ldap_opts.h b/src/providers/ldap/ldap_opts.h
> index 0a33289..61e3309 100644
> --- a/src/providers/ldap/ldap_opts.h
> +++ b/src/providers/ldap/ldap_opts.h
> @@ -155,7 +155,7 @@ struct sdap_attr_map rfc2307_user_map[] = {
>     { "ldap_user_principal", "krbPrincipalName", SYSDB_UPN, NULL },
>     { "ldap_user_fullname", "cn", SYSDB_FULLNAME, NULL },
>     { "ldap_user_member_of", NULL, SYSDB_MEMBEROF, NULL },
> { "ldap_user_objectsid", NULL, SYSDB_SID, NULL },
> { "ldap_user_objectsid", "objectSID", SYSDB_SID, NULL },
> { "ldap_user_primary_group", NULL, SYSDB_PRIMARY_GROUP, NULL },
> { "ldap_user_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> { "ldap_user_entry_usn", NULL, SYSDB_USN, NULL },
>
> @@ -188,7 +188,7 @@ struct sdap_attr_map rfc2307_group_map[] = {
>     { "ldap_group_pwd", "userPassword", SYSDB_PWD, NULL },
>     { "ldap_group_gid_number", "gidNumber", SYSDB_GIDNUM, NULL },
>     { "ldap_group_member", "memberuid", SYSDB_MEMBER, NULL },
> { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
>
> @@ -207,7 +207,7 @@ struct sdap_attr_map rfc2307bis_user_map[] = {
>     { "ldap_user_principal", "krbPrincipalName", SYSDB_UPN, NULL },
>     { "ldap_user_fullname", "cn", SYSDB_FULLNAME, NULL },
>     { "ldap_user_member_of", "memberOf", SYSDB_MEMBEROF, NULL },
> { "ldap_user_objectsid", NULL, SYSDB_SID, NULL },
> { "ldap_user_objectsid", "objectSID", SYSDB_SID, NULL },
> { "ldap_user_primary_group", NULL, SYSDB_PRIMARY_GROUP, NULL },
> { "ldap_user_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> { "ldap_user_entry_usn", NULL, SYSDB_USN, NULL },
>
> @@ -240,7 +240,7 @@ struct sdap_attr_map rfc2307bis_group_map[] = {
>     { "ldap_group_pwd", "userPassword", SYSDB_PWD, NULL },
>     { "ldap_group_gid_number", "gidNumber", SYSDB_GIDNUM, NULL },
>     { "ldap_group_member", "member", SYSDB_MEMBER, NULL },
> { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },

These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383

Dereference failed with openldapserver.
[sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
[sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
[sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
[sdap_print_server] (0x2000): Searching 10.16.64.191
[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
[sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
[sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
[sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
[generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
[sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
[sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
[sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]

SSSD did equivalen of next command line search:
[root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
    cn=topgrp
Protocol error (2)
Additional information: Dereference control: attribute decoding error

Dereferencing failed because attribute objectSID isn't in any objectclass on
openldap server.

Summary:
We should revert this patch. ObjectSID should not be set by default
in rfc2307 or rfc2307bis scheme.

LS

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Lukas Slebodnik]

On (24/10/14 14:39), Lukas Slebodnik wrote:
> On (29/09/14 17:58), Michal Židek wrote:
> > Hi,
> >
> > this patch solves the ticket:
> > https://fedorahosted.org/sssd/ticket/2361
> >
> > From the discussion on bugzilla and the ticket comments, it
> > looks like Jakub and Lukas were for changes in documentation
> > only (+ maybe warning/error if user misconfigured SSSD).
> >
> > I do not want look like I ignore the opinions of others
> > but I think the best way is to simply change the defaults
> > to the same as expected by AD backend. While I do agree
> > that users should set their id_provider option to AD
> > if they use AD backend, this option is more or less
> > AD specific, so I think it would be better to have
> > the same defaults with id_provider ldap. This
> > would also solve the small regressions that are
> > mentioned in the bugzilla (by users who do use
> > id_provider=AD and id_mapping=true).
> >
> > OTOH if this approach is disliked by other developers
> > I have no big problems with the man page + warning/error
> > solution.
> >
> > Simple patch is attached.
> >
> > Thanks,
> > Michal
> > From 2d0f2f3c62eefe4f55cb21b91538bad8b68f6df5 Mon Sep 17 00:00:00 2001
> > From: Michal Zidek 
> > Date: Mon, 29 Sep 2014 17:17:59 +0200
> > Subject: [PATCH] LDAP: Change defaults for ldap_user/group_objectsid
> >
> > Fixes:
> > https://fedorahosted.org/sssd/ticket/2361
> >
> > src/man/sssd-ldap.5.xml        | 4 ++--
> > src/providers/ldap/ldap_opts.h | 8 ++++----
> > 2 files changed, 6 insertions(+), 6 deletions(-)
> >
> > diff --git a/src/man/sssd-ldap.5.xml b/src/man/sssd-ldap.5.xml
> > index 03ea794..a21ffc1 100644
> > --- a/src/man/sssd-ldap.5.xml
> > +++ b/src/man/sssd-ldap.5.xml
> > @@ -346,7 +346,7 @@
> >                             necessary for ActiveDirectory servers.
> >                         </para>
> >                         <para>
> >                        Default: objectSid for ActiveDirectory, not set
> >                        Default: ipaNTSecurityIdentifier for IPA, objectSID
> >                       for other servers.
> >                   </para>
> >               </listitem>
> >
> > @@ -854,7 +854,7 @@
> >                             necessary for ActiveDirectory servers.
> >                         </para>
> >                         <para>
> >                        Default: objectSid for ActiveDirectory, not set
> >                        Default: ipaNTSecurityIdentifier for IPA, objectSID
> >                       for other servers.
> >                   </para>
> >               </listitem>
> >
> > diff --git a/src/providers/ldap/ldap_opts.h b/src/providers/ldap/ldap_opts.h
> > index 0a33289..61e3309 100644
> > --- a/src/providers/ldap/ldap_opts.h
> > +++ b/src/providers/ldap/ldap_opts.h
> > @@ -155,7 +155,7 @@ struct sdap_attr_map rfc2307_user_map[] = {
> >     { "ldap_user_principal", "krbPrincipalName", SYSDB_UPN, NULL },
> >     { "ldap_user_fullname", "cn", SYSDB_FULLNAME, NULL },
> >     { "ldap_user_member_of", NULL, SYSDB_MEMBEROF, NULL },
> > { "ldap_user_objectsid", NULL, SYSDB_SID, NULL },
> > { "ldap_user_objectsid", "objectSID", SYSDB_SID, NULL },
> > { "ldap_user_primary_group", NULL, SYSDB_PRIMARY_GROUP, NULL },
> > { "ldap_user_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > { "ldap_user_entry_usn", NULL, SYSDB_USN, NULL },
> >
> > @@ -188,7 +188,7 @@ struct sdap_attr_map rfc2307_group_map[] = {
> >     { "ldap_group_pwd", "userPassword", SYSDB_PWD, NULL },
> >     { "ldap_group_gid_number", "gidNumber", SYSDB_GIDNUM, NULL },
> >     { "ldap_group_member", "memberuid", SYSDB_MEMBER, NULL },
> > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> >
> > @@ -207,7 +207,7 @@ struct sdap_attr_map rfc2307bis_user_map[] = {
> >     { "ldap_user_principal", "krbPrincipalName", SYSDB_UPN, NULL },
> >     { "ldap_user_fullname", "cn", SYSDB_FULLNAME, NULL },
> >     { "ldap_user_member_of", "memberOf", SYSDB_MEMBEROF, NULL },
> > { "ldap_user_objectsid", NULL, SYSDB_SID, NULL },
> > { "ldap_user_objectsid", "objectSID", SYSDB_SID, NULL },
> > { "ldap_user_primary_group", NULL, SYSDB_PRIMARY_GROUP, NULL },
> > { "ldap_user_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > { "ldap_user_entry_usn", NULL, SYSDB_USN, NULL },
> >
> > @@ -240,7 +240,7 @@ struct sdap_attr_map rfc2307bis_group_map[] = {
> >     { "ldap_group_pwd", "userPassword", SYSDB_PWD, NULL },
> >     { "ldap_group_gid_number", "gidNumber", SYSDB_GIDNUM, NULL },
> >     { "ldap_group_member", "member", SYSDB_MEMBER, NULL },
> > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
>
> These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
>
> Dereference failed with openldapserver.
> [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> [sdap_print_server] (0x2000): Searching 10.16.64.191
> [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
>
> SSSD did equivalen of next command line search:
> [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
>    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
>    cn=topgrp
> Protocol error (2)
> Additional information: Dereference control: attribute decoding error
>
> Dereferencing failed because attribute objectSID isn't in any objectclass on
> openldap server.
>
> Summary:
> We should revert this patch. ObjectSID should not be set by default
> in rfc2307 or rfc2307bis scheme.

BTW we had the same problem with nonstandard UUID attribute
https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...

LS

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Jakub Hrozek]

On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> >
> > These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> >
> > Dereference failed with openldapserver.
> > [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> >
> > SSSD did equivalen of next command line search:
> > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> >    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> >    cn=topgrp
> > Protocol error (2)
> > Additional information: Dereference control: attribute decoding error
> >
> > Dereferencing failed because attribute objectSID isn't in any objectclass on
> > openldap server.
> >
> > Summary:
> > We should revert this patch. ObjectSID should not be set by default
> > in rfc2307 or rfc2307bis scheme.
>
> BTW we had the same problem with nonstandard UUID attribute
> https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
>
> LS

OK, this is bad, thanks a lot for catching the problem.

I agree that we should revert the patch as a first step. But at the same
time, the changes in 1.11 caused problems for several users running with
id_provider=ldap enrolled with an AD server.

In retrospective, I wish we limited some of the AD-specific features to
the AD provider only, but in order to mitigate the problem, what about
setting the defaults if (and only if) we detect the server is an AD
server from rootDSE? We already do that for tokenGroups..

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Michal Židek]

On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > >
> > > These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > >
> > > Dereference failed with openldapserver.
> > > [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > >
> > > SSSD did equivalen of next command line search:
> > > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > >     -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > >     cn=topgrp
> > > Protocol error (2)
> > > Additional information: Dereference control: attribute decoding error
> > >
> > > Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > openldap server.
> > >
> > > Summary:
> > > We should revert this patch. ObjectSID should not be set by default
> > > in rfc2307 or rfc2307bis scheme.
> >
> > BTW we had the same problem with nonstandard UUID attribute
> > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> >
> > LS
>
> OK, this is bad, thanks a lot for catching the problem.
>
> I agree that we should revert the patch as a first step. But at the same
> time, the changes in 1.11 caused problems for several users running with
> id_provider=ldap enrolled with an AD server.
>
> In retrospective, I wish we limited some of the AD-specific features to
> the AD provider only, but in order to mitigate the problem, what about
> setting the defaults if (and only if) we detect the server is an AD
> server from rootDSE? We already do that for tokenGroups..

After discussion with Lukas I think we should not do this.
Not even for tokenGroups. If user specifies his provider
as ldap, we should not do background magic that changes
defaults behind the scenes (based on rootDSE or anything
else).

Errors like the ones that my original (regression making)
patch tried to solve (id_mapping is set but ldap_*_objectsid
is not) should be detected by config file validator (once we
have it). Adding hacks to the code is not a good way (so
this patch was not good as well).

Michal

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Jakub Hrozek]

On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > >
> > > > These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > >
> > > > Dereference failed with openldapserver.
> > > > [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > >
> > > > SSSD did equivalen of next command line search:
> > > > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > >    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > >    cn=topgrp
> > > > Protocol error (2)
> > > > Additional information: Dereference control: attribute decoding error
> > > >
> > > > Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > openldap server.
> > > >
> > > > Summary:
> > > > We should revert this patch. ObjectSID should not be set by default
> > > > in rfc2307 or rfc2307bis scheme.
> > >
> > > BTW we had the same problem with nonstandard UUID attribute
> > > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > >
> > > LS
> >
> > OK, this is bad, thanks a lot for catching the problem.
> >
> > I agree that we should revert the patch as a first step. But at the same
> > time, the changes in 1.11 caused problems for several users running with
> > id_provider=ldap enrolled with an AD server.
> >
> > In retrospective, I wish we limited some of the AD-specific features to
> > the AD provider only, but in order to mitigate the problem, what about
> > setting the defaults if (and only if) we detect the server is an AD
> > server from rootDSE? We already do that for tokenGroups..
>
> After discussion with Lukas I think we should not do this.
> Not even for tokenGroups. If user specifies his provider
> as ldap, we should not do background magic that changes
> defaults behind the scenes (based on rootDSE or anything
> else).

I totally agree, see my previous mail -- "In retrospective, I wish we
limited some of the AD-specific features to the AD provider only".

But I don't think we can do backwards-incompatible changes so easily
now. It's completely out of question for a minor update (1.12.x) and must
be handled with extreme care in major update (1.13).

Additionally, we already detect search bases and other config from
rootDSE, what makes the default SID values any different?
> Errors like the ones that my original (regression making)
> patch tried to solve (id_mapping is set but ldap_*_objectsid
> is not) should be detected by config file validator (once we
> have it). Adding hacks to the code is not a good way (so
> this patch was not good as well).

Yes, but what is even worse is users running a supported and valid
configuration and having their setup broken with a yum upgrade.

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Lukas Slebodnik]

On (24/10/14 21:14), Jakub Hrozek wrote:
> On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > >
> > > > > These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > >
> > > > > Dereference failed with openldapserver.
> > > > > [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > >
> > > > > SSSD did equivalen of next command line search:
> > > > > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > >    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > >    cn=topgrp
> > > > > Protocol error (2)
> > > > > Additional information: Dereference control: attribute decoding error
> > > > >
> > > > > Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > openldap server.
> > > > >
> > > > > Summary:
> > > > > We should revert this patch. ObjectSID should not be set by default
> > > > > in rfc2307 or rfc2307bis scheme.
> > > >
> > > > BTW we had the same problem with nonstandard UUID attribute
> > > > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > >
> > > > LS
> > >
> > > OK, this is bad, thanks a lot for catching the problem.
> > >
> > > I agree that we should revert the patch as a first step. But at the same
> > > time, the changes in 1.11 caused problems for several users running with
> > > id_provider=ldap enrolled with an AD server.
> > >
> > > In retrospective, I wish we limited some of the AD-specific features to
> > > the AD provider only, but in order to mitigate the problem, what about
> > > setting the defaults if (and only if) we detect the server is an AD
> > > server from rootDSE? We already do that for tokenGroups..
> >
> > After discussion with Lukas I think we should not do this.
> > Not even for tokenGroups. If user specifies his provider
> > as ldap, we should not do background magic that changes
> > defaults behind the scenes (based on rootDSE or anything
> > else).
>
> I totally agree, see my previous mail -- "In retrospective, I wish we
> limited some of the AD-specific features to the AD provider only".
>
> But I don't think we can do backwards-incompatible changes so easily
> now. It's completely out of question for a minor update (1.12.x) and must
> be handled with extreme care in major update (1.13).
>
> Additionally, we already detect search bases and other config from
> rootDSE, what makes the default SID values any different?

The problem was automatic enabling of features based on rootDSE.
User used ldap provider with AD but without idmapping. The root of problem was
that tokengroups were enabled by default (based on rootDSE) and SIDs could not
be downloaded. Even if they were downloaded id mapping is disabled
It would not spped upa anything.

We tried to solve it with this patch, but there is another solution.
To disable tokengroups for ldap provider (rfc2307, rfc2307bis).

If someone wants to use AD as any other ldap server we should not enable ad
features by default. If someone wants extra ad features (tokengroups, id
mapping) then they can use ad provider or ldap provider and enable all options
manually. We should not be smart in ldap provider. The smart provider(s)
is(are) ad(,ipa).

LS

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Jakub Hrozek]

On Sat, Oct 25, 2014 at 11:59:36PM +0200, Lukas Slebodnik wrote:
> On (24/10/14 21:14), Jakub Hrozek wrote:
> > On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > >-    { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > >+    { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > >     { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > >     { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > >     { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > > > These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > > which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > > >
> > > > > > Dereference failed with openldapserver.
> > > > > > [sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > > [sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > > [sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > > [sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > > [sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > > [sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > > [sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > > [sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > > [sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > > >
> > > > > > SSSD did equivalen of next command line search:
> > > > > > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > > >    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > > >    cn=topgrp
> > > > > > Protocol error (2)
> > > > > > Additional information: Dereference control: attribute decoding error
> > > > > >
> > > > > > Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > > openldap server.
> > > > > >
> > > > > > Summary:
> > > > > > We should revert this patch. ObjectSID should not be set by default
> > > > > > in rfc2307 or rfc2307bis scheme.
> > > > >
> > > > > BTW we had the same problem with nonstandard UUID attribute
> > > > > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > > >
> > > > > LS
> > > >
> > > > OK, this is bad, thanks a lot for catching the problem.
> > > >
> > > > I agree that we should revert the patch as a first step. But at the same
> > > > time, the changes in 1.11 caused problems for several users running with
> > > > id_provider=ldap enrolled with an AD server.
> > > >
> > > > In retrospective, I wish we limited some of the AD-specific features to
> > > > the AD provider only, but in order to mitigate the problem, what about
> > > > setting the defaults if (and only if) we detect the server is an AD
> > > > server from rootDSE? We already do that for tokenGroups..
> > >
> > > After discussion with Lukas I think we should not do this.
> > > Not even for tokenGroups. If user specifies his provider
> > > as ldap, we should not do background magic that changes
> > > defaults behind the scenes (based on rootDSE or anything
> > > else).
> >
> > I totally agree, see my previous mail -- "In retrospective, I wish we
> > limited some of the AD-specific features to the AD provider only".
> >
> > But I don't think we can do backwards-incompatible changes so easily
> > now. It's completely out of question for a minor update (1.12.x) and must
> > be handled with extreme care in major update (1.13).
> >
> > Additionally, we already detect search bases and other config from
> > rootDSE, what makes the default SID values any different?
>
> The problem was automatic enabling of features based on rootDSE.
> User used ldap provider with AD but without idmapping. The root of problem was
> that tokengroups were enabled by default (based on rootDSE) and SIDs could not
> be downloaded. Even if they were downloaded id mapping is disabled
> It would not spped upa anything.

It wouldn't speed up anything for the first run, true, but there is a
big potential benefit for lookups of other users.

Consider Alice and Bob are both members of two same groups. Alice logs
in first, the cache is empty, so we look up both sids. So we have a base
search for the user, a base search for the tokenGroups and then two searches
based on SID. Then Bob comes in, we check his tokenGroups attribute and
see that both SIDs are already cached, so we just link Bob's user entry
with the SIDs and we're done -- total of two lookups with scope=base.

So this change /does/ have a nice speed benefit, however, see below..
> We tried to solve it with this patch, but there is another solution.
> To disable tokengroups for ldap provider (rfc2307, rfc2307bis).

I spent pretty much the whole day today triaging issues after users
upgraded to RHEL-6.6. And I tend to agree we should revert this change
for id_provider=ldap && ldap_id_mapping=False.

Setting the SID attribute values to the AD defaults as I proposed earlier
is still something that might make sense and would solve maybe 50% of
issues. But the other 50% are strange errors like looking
up SIDs from another subdomain, where the LDAP provider has no access etc.
> If someone wants to use AD as any other ldap server we should not enable ad
> features by default.

^^^^^^^^
I disagree, we should take advantage of whatever features we have at
our disposal. But tokenGroups along with the LDAP provider proved to be
too fragile..

So after experiencing the breakage from downstream, I also vote for
enabling tokenGroups if:
    * id_provider = ad, with or without ID mapping
    * ldap_provider = ldap, without ID mapping

To make sure id_provider=ldap && mapping=true we should also set the user
and group SID attribute defaults.

Thoughts?
> If someone wants extra ad features (tokengroups, id
> mapping) then they can use ad provider or ldap provider and enable all options
> manually. We should not be smart in ldap provider. The smart provider(s)
> is(are) ad(,ipa).
>
> LS
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Lukas Slebodnik]

On (03/11/14 17:10), Jakub Hrozek wrote:
> On Sat, Oct 25, 2014 at 11:59:36PM +0200, Lukas Slebodnik wrote:
> > On (24/10/14 21:14), Jakub Hrozek wrote:
> > > On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > > > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > > > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > >>-    { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > >>+    { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > >>     { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > >>     { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > >>     { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > > > >These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > > >which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > > > >
> > > > > > >Dereference failed with openldapserver.
> > > > > > >[sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > > >[sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > > >[sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > > >[sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > > >[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > > >[sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > > >[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > > >[sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >[sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >[generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > > >[sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > > >[sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > > >[sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > > > >
> > > > > > >
> > > > > > >SSSD did equivalen of next command line search:
> > > > > > >[root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > > > >    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > > > >    cn=topgrp
> > > > > > >Protocol error (2)
> > > > > > >Additional information: Dereference control: attribute decoding error
> > > > > > >
> > > > > > >Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > > >openldap server.
> > > > > > >
> > > > > > >Summary:
> > > > > > >We should revert this patch. ObjectSID should not be set by default
> > > > > > >in rfc2307 or rfc2307bis scheme.
> > > > > > >
> > > > > > BTW we had the same problem with nonstandard UUID attribute
> > > > > > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > > > >
> > > > > > LS
> > > > >
> > > > > OK, this is bad, thanks a lot for catching the problem.
> > > > >
> > > > > I agree that we should revert the patch as a first step. But at the same
> > > > > time, the changes in 1.11 caused problems for several users running with
> > > > > id_provider=ldap enrolled with an AD server.
> > > > >
> > > > > In retrospective, I wish we limited some of the AD-specific features to
> > > > > the AD provider only, but in order to mitigate the problem, what about
> > > > > setting the defaults if (and only if) we detect the server is an AD
> > > > > server from rootDSE? We already do that for tokenGroups..
> > > >
> > > > After discussion with Lukas I think we should not do this.
> > > > Not even for tokenGroups. If user specifies his provider
> > > > as ldap, we should not do background magic that changes
> > > > defaults behind the scenes (based on rootDSE or anything
> > > > else).
> > >
> > > I totally agree, see my previous mail -- "In retrospective, I wish we
> > > limited some of the AD-specific features to the AD provider only".
> > >
> > > But I don't think we can do backwards-incompatible changes so easily
> > > now. It's completely out of question for a minor update (1.12.x) and must
> > > be handled with extreme care in major update (1.13).
> > >
> > > Additionally, we already detect search bases and other config from
> > > rootDSE, what makes the default SID values any different?
> >
> > The problem was automatic enabling of features based on rootDSE.
> > User used ldap provider with AD but without idmapping. The root of problem was
> > that tokengroups were enabled by default (based on rootDSE) and SIDs could not
> > be downloaded. Even if they were downloaded id mapping is disabled
> > It would not spped upa anything.
>
> It wouldn't speed up anything for the first run, true, but there is a
> big potential benefit for lookups of other users.
>
> Consider Alice and Bob are both members of two same groups. Alice logs
> in first, the cache is empty, so we look up both sids. So we have a base
> search for the user, a base search for the tokenGroups and then two searches
> based on SID. Then Bob comes in, we check his tokenGroups attribute and
> see that both SIDs are already cached, so we just link Bob's user entry
> with the SIDs and we're done -- total of two lookups with scope=base.
>
> So this change /does/ have a nice speed benefit, however, see below..
> > We tried to solve it with this patch, but there is another solution.
> > To disable tokengroups for ldap provider (rfc2307, rfc2307bis).
>
> I spent pretty much the whole day today triaging issues after users
> upgraded to RHEL-6.6. And I tend to agree we should revert this change
> for id_provider=ldap && ldap_id_mapping=False.
>
> Setting the SID attribute values to the AD defaults as I proposed earlier
> is still something that might make sense and would solve maybe 50% of
> issues. But the other 50% are strange errors like looking
> up SIDs from another subdomain, where the LDAP provider has no access etc.
> > If someone wants to use AD as any other ldap server we should not enable ad
> > features by default.
>
> ^^^^^^^^
> I disagree, we should take advantage of whatever features we have at
> our disposal. But tokenGroups along with the LDAP provider proved to be
> too fragile..
>
> So after experiencing the breakage from downstream, I also vote for
> enabling tokenGroups if:
> id_provider = ad, with or without ID mapping
> ldap_provider = ldap, without ID mapping
>
> To make sure id_provider=ldap && mapping=true we should also set the user
> and group SID attribute defaults.

It is clear that tokengroups and id mapping are related options.

I would prefer to merge boolean options ldap_id_mapping, ldap_use_tokengroups
(and deprecated one option).

IMHO, it is better solutions than auto-enabling features based on rootDSE.

LS

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Jakub Hrozek]

On Mon, Nov 03, 2014 at 07:43:22PM +0100, Lukas Slebodnik wrote:
> On (03/11/14 17:10), Jakub Hrozek wrote:
> > On Sat, Oct 25, 2014 at 11:59:36PM +0200, Lukas Slebodnik wrote:
> > > On (24/10/14 21:14), Jakub Hrozek wrote:
> > > > On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > > > > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > > > > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > >>>-    { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > >>>+    { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > >>>     { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > >>>     { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > >>>     { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > > > >>These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > > >>which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > > > >>
> > > > > > >>Dereference failed with openldapserver.
> > > > > > >>[sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > > >>[sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > > >>[sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > > >>[sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > > >>[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > > >>[sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > > >>[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > > >>[sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >>[sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >>[generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > > >>[sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > > >>[sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > > >>[sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > > > >>
> > > > > > >>
> > > > > > >>SSSD did equivalen of next command line search:
> > > > > > >>[root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > > > >>    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > > > >>    cn=topgrp
> > > > > > >>Protocol error (2)
> > > > > > >>Additional information: Dereference control: attribute decoding error
> > > > > > >>
> > > > > > >>Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > > >>openldap server.
> > > > > > >>
> > > > > > >>Summary:
> > > > > > >>We should revert this patch. ObjectSID should not be set by default
> > > > > > >>in rfc2307 or rfc2307bis scheme.
> > > > > > >>
> > > > > > >BTW we had the same problem with nonstandard UUID attribute
> > > > > > >https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > > > > >
> > > > > > >LS
> > > > > >
> > > > > > OK, this is bad, thanks a lot for catching the problem.
> > > > > >
> > > > > > I agree that we should revert the patch as a first step. But at the same
> > > > > > time, the changes in 1.11 caused problems for several users running with
> > > > > > id_provider=ldap enrolled with an AD server.
> > > > > >
> > > > > > In retrospective, I wish we limited some of the AD-specific features to
> > > > > > the AD provider only, but in order to mitigate the problem, what about
> > > > > > setting the defaults if (and only if) we detect the server is an AD
> > > > > > server from rootDSE? We already do that for tokenGroups..
> > > > >
> > > > > After discussion with Lukas I think we should not do this.
> > > > > Not even for tokenGroups. If user specifies his provider
> > > > > as ldap, we should not do background magic that changes
> > > > > defaults behind the scenes (based on rootDSE or anything
> > > > > else).
> > > >
> > > > I totally agree, see my previous mail -- "In retrospective, I wish we
> > > > limited some of the AD-specific features to the AD provider only".
> > > >
> > > > But I don't think we can do backwards-incompatible changes so easily
> > > > now. It's completely out of question for a minor update (1.12.x) and must
> > > > be handled with extreme care in major update (1.13).
> > > >
> > > > Additionally, we already detect search bases and other config from
> > > > rootDSE, what makes the default SID values any different?
> > >
> > > The problem was automatic enabling of features based on rootDSE.
> > > User used ldap provider with AD but without idmapping. The root of problem was
> > > that tokengroups were enabled by default (based on rootDSE) and SIDs could not
> > > be downloaded. Even if they were downloaded id mapping is disabled
> > > It would not spped upa anything.
> >
> > It wouldn't speed up anything for the first run, true, but there is a
> > big potential benefit for lookups of other users.
> >
> > Consider Alice and Bob are both members of two same groups. Alice logs
> > in first, the cache is empty, so we look up both sids. So we have a base
> > search for the user, a base search for the tokenGroups and then two searches
> > based on SID. Then Bob comes in, we check his tokenGroups attribute and
> > see that both SIDs are already cached, so we just link Bob's user entry
> > with the SIDs and we're done -- total of two lookups with scope=base.
> >
> > So this change /does/ have a nice speed benefit, however, see below..
> > > We tried to solve it with this patch, but there is another solution.
> > > To disable tokengroups for ldap provider (rfc2307, rfc2307bis).
> >
> > I spent pretty much the whole day today triaging issues after users
> > upgraded to RHEL-6.6. And I tend to agree we should revert this change
> > for id_provider=ldap && ldap_id_mapping=False.
> >
> > Setting the SID attribute values to the AD defaults as I proposed earlier
> > is still something that might make sense and would solve maybe 50% of
> > issues. But the other 50% are strange errors like looking
> > up SIDs from another subdomain, where the LDAP provider has no access etc.
> > > If someone wants to use AD as any other ldap server we should not enable ad
> > > features by default.
> >
> > ^^^^^^^^
> > I disagree, we should take advantage of whatever features we have at
> > our disposal. But tokenGroups along with the LDAP provider proved to be
> > too fragile..
> >
> > So after experiencing the breakage from downstream, I also vote for
> > enabling tokenGroups if:
> > id_provider = ad, with or without ID mapping
> > ldap_provider = ldap, without ID mapping
> >
> > To make sure id_provider=ldap && mapping=true we should also set the user
> > and group SID attribute defaults.
>
> It is clear that tokengroups and id mapping are related options.

They are not. As I said above, tokengroups can be used to speed up POSIX
lookups, too.
> I would prefer to merge boolean options ldap_id_mapping, ldap_use_tokengroups
> (and deprecated one option).
>
> IMHO, it is better solutions than auto-enabling features based on rootDSE.

I disagree.

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Lukas Slebodnik]

On (03/11/14 19:51), Jakub Hrozek wrote:
> On Mon, Nov 03, 2014 at 07:43:22PM +0100, Lukas Slebodnik wrote:
> > On (03/11/14 17:10), Jakub Hrozek wrote:
> > > On Sat, Oct 25, 2014 at 11:59:36PM +0200, Lukas Slebodnik wrote:
> > > > On (24/10/14 21:14), Jakub Hrozek wrote:
> > > > > On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > > > > > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > > > > >On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > >>>>-    { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > >>>>+    { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > >>>>     { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > >>>>     { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > >>>>     { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > > > >>>These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > > >>>which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > > > >>>
> > > > > > >>>Dereference failed with openldapserver.
> > > > > > >>>[sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > > >>>[sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > > >>>[sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > > >>>[sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > > >>>[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > > >>>[sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > > >>>[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > > >>>[sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >>>[sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > > >>>[generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > > >>>[sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > > >>>[sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > > >>>[sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>SSSD did equivalen of next command line search:
> > > > > > >>>[root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > > > >>>    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > > > >>>    cn=topgrp
> > > > > > >>>Protocol error (2)
> > > > > > >>>Additional information: Dereference control: attribute decoding error
> > > > > > >>>
> > > > > > >>>Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > > >>>openldap server.
> > > > > > >>>
> > > > > > >>>Summary:
> > > > > > >>>We should revert this patch. ObjectSID should not be set by default
> > > > > > >>>in rfc2307 or rfc2307bis scheme.
> > > > > > >>>
> > > > > > >>BTW we had the same problem with nonstandard UUID attribute
> > > > > > >>https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > > > > >>
> > > > > > >>LS
> > > > > > >
> > > > > > >OK, this is bad, thanks a lot for catching the problem.
> > > > > > >
> > > > > > >I agree that we should revert the patch as a first step. But at the same
> > > > > > >time, the changes in 1.11 caused problems for several users running with
> > > > > > >id_provider=ldap enrolled with an AD server.
> > > > > > >
> > > > > > >In retrospective, I wish we limited some of the AD-specific features to
> > > > > > >the AD provider only, but in order to mitigate the problem, what about
> > > > > > >setting the defaults if (and only if) we detect the server is an AD
> > > > > > >server from rootDSE? We already do that for tokenGroups..
> > > > > >
> > > > > > After discussion with Lukas I think we should not do this.
> > > > > > Not even for tokenGroups. If user specifies his provider
> > > > > > as ldap, we should not do background magic that changes
> > > > > > defaults behind the scenes (based on rootDSE or anything
> > > > > > else).
> > > > >
> > > > > I totally agree, see my previous mail -- "In retrospective, I wish we
> > > > > limited some of the AD-specific features to the AD provider only".
> > > > >
> > > > > But I don't think we can do backwards-incompatible changes so easily
> > > > > now. It's completely out of question for a minor update (1.12.x) and must
> > > > > be handled with extreme care in major update (1.13).
> > > > >
> > > > > Additionally, we already detect search bases and other config from
> > > > > rootDSE, what makes the default SID values any different?
> > > >
> > > > The problem was automatic enabling of features based on rootDSE.
> > > > User used ldap provider with AD but without idmapping. The root of problem was
> > > > that tokengroups were enabled by default (based on rootDSE) and SIDs could not
> > > > be downloaded. Even if they were downloaded id mapping is disabled
> > > > It would not spped upa anything.
> > >
> > > It wouldn't speed up anything for the first run, true, but there is a
> > > big potential benefit for lookups of other users.
> > >
> > > Consider Alice and Bob are both members of two same groups. Alice logs
> > > in first, the cache is empty, so we look up both sids. So we have a base
> > > search for the user, a base search for the tokenGroups and then two searches
> > > based on SID. Then Bob comes in, we check his tokenGroups attribute and
> > > see that both SIDs are already cached, so we just link Bob's user entry
> > > with the SIDs and we're done -- total of two lookups with scope=base.
> > >
> > > So this change /does/ have a nice speed benefit, however, see below..

there's nothing about POSIX lookups.
> > > > We tried to solve it with this patch, but there is another solution.
> > > > To disable tokengroups for ldap provider (rfc2307, rfc2307bis).
> > >
> > > I spent pretty much the whole day today triaging issues after users
> > > upgraded to RHEL-6.6. And I tend to agree we should revert this change
> > > for id_provider=ldap && ldap_id_mapping=False.
> > >
> > > Setting the SID attribute values to the AD defaults as I proposed earlier
> > > is still something that might make sense and would solve maybe 50% of
> > > issues. But the other 50% are strange errors like looking
> > > up SIDs from another subdomain, where the LDAP provider has no access etc.

there's nothing about POSIX lookups either.
> > > > If someone wants to use AD as any other ldap server we should not enable ad
> > > > features by default.
> > >
> > > ^^^^^^^^
> > > I disagree, we should take advantage of whatever features we have at
> > > our disposal. But tokenGroups along with the LDAP provider proved to be
> > > too fragile..
> > >
> > > So after experiencing the breakage from downstream, I also vote for
> > > enabling tokenGroups if:
> > > id_provider = ad, with or without ID mapping
> > > ldap_provider = ldap, without ID mapping
> > >
> > > To make sure id_provider=ldap && mapping=true we should also set the user
> > > and group SID attribute defaults.
> >
> > It is clear that tokengroups and id mapping are related options.
>
> They are not. As I said above, tokengroups can be used to speed up POSIX
> lookups, too.

I read mails few times, but I didn't find anything about POSIX lookups.
Did I overlook something?

LS

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Jakub Hrozek]

On Mon, Nov 03, 2014 at 08:18:06PM +0100, Lukas Slebodnik wrote:
> On (03/11/14 19:51), Jakub Hrozek wrote:
> > On Mon, Nov 03, 2014 at 07:43:22PM +0100, Lukas Slebodnik wrote:
> > > On (03/11/14 17:10), Jakub Hrozek wrote:
> > > > On Sat, Oct 25, 2014 at 11:59:36PM +0200, Lukas Slebodnik wrote:
> > > > > On (24/10/14 21:14), Jakub Hrozek wrote:
> > > > > > On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > > > > > > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > > > > > >On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > > >>>>-    { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > > >>>>+    { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > > >>>>     { "ldap_group_modify_timestamp", "modifyTimestamp", SYSDB_ORIG_MODSTAMP, NULL },
> > > > > > > >>>>     { "ldap_group_entry_usn", NULL, SYSDB_USN, NULL },
> > > > > > > >>>>     { "ldap_group_type", NULL, SYSDB_GROUP_TYPE, NULL },
> > > > > > > >>>These changes in rfc2307_user_map and rfc2307bis_user_map caused regression,
> > > > > > > >>>which was fixed by ticket https://fedorahosted.org/sssd/ticket/2383
> > > > > > > >>>
> > > > > > > >>>Dereference failed with openldapserver.
> > > > > > > >>>[sdap_nested_group_process_send] (0x2000): Dereferencing members of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > > > >>>[sdap_deref_search_send] (0x2000): Server supports OpenLDAP deref
> > > > > > > >>>[sdap_x_deref_search_send] (0x0400): Dereferencing entry [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > > > >>>[sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > > > >>>[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectClass]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [cn]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [userPassword]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [gidNumber]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [member]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [objectSID]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [modifyTimestamp]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x1000): Requesting attrs: [uid]
> > > > > > > >>>[sdap_get_generic_ext_step] (0x2000): ldap_search_ext called, msgid = 3
> > > > > > > >>>[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
> > > > > > > >>>[sdap_get_generic_op_finished] (0x0400): Search result: Protocol error(2), Dereference control: attribute decoding error
> > > > > > > >>>[sdap_get_generic_op_finished] (0x0040): Unexpected result from ldap: Protocol error(2), Dereference control: attribute decoding error
> > > > > > > >>>[generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv failed [5]: Input/output error
> > > > > > > >>>[sdap_deref_search_done] (0x0040): dereference processing failed [5]: Input/output error
> > > > > > > >>>[sdap_nested_group_deref_direct_done] (0x0020): Error processing direct membership [5]: Input/output error
> > > > > > > >>>[sdap_nested_done] (0x0020): Nested group processing failed: [5][Input/output error]
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>SSSD did equivalen of next command line search:
> > > > > > > >>>[root@host ~]# ldapsearch -x -LLL -h $SERVER -b 'dc=example,dc=com' \
> > > > > > > >>>    -E '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid' \
> > > > > > > >>>    cn=topgrp
> > > > > > > >>>Protocol error (2)
> > > > > > > >>>Additional information: Dereference control: attribute decoding error
> > > > > > > >>>
> > > > > > > >>>Dereferencing failed because attribute objectSID isn't in any objectclass on
> > > > > > > >>>openldap server.
> > > > > > > >>>
> > > > > > > >>>Summary:
> > > > > > > >>>We should revert this patch. ObjectSID should not be set by default
> > > > > > > >>>in rfc2307 or rfc2307bis scheme.
> > > > > > > >>>
> > > > > > > >>BTW we had the same problem with nonstandard UUID attribute
> > > > > > > >>https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > > > > > >>
> > > > > > > >>LS
> > > > > > > >
> > > > > > > >OK, this is bad, thanks a lot for catching the problem.
> > > > > > > >
> > > > > > > >I agree that we should revert the patch as a first step. But at the same
> > > > > > > >time, the changes in 1.11 caused problems for several users running with
> > > > > > > >id_provider=ldap enrolled with an AD server.
> > > > > > > >
> > > > > > > >In retrospective, I wish we limited some of the AD-specific features to
> > > > > > > >the AD provider only, but in order to mitigate the problem, what about
> > > > > > > >setting the defaults if (and only if) we detect the server is an AD
> > > > > > > >server from rootDSE? We already do that for tokenGroups..
> > > > > > > 
> > > > > > > After discussion with Lukas I think we should not do this.
> > > > > > > Not even for tokenGroups. If user specifies his provider
> > > > > > > as ldap, we should not do background magic that changes
> > > > > > > defaults behind the scenes (based on rootDSE or anything
> > > > > > > else).
> > > > > >
> > > > > > I totally agree, see my previous mail -- "In retrospective, I wish we
> > > > > > limited some of the AD-specific features to the AD provider only".
> > > > > >
> > > > > > But I don't think we can do backwards-incompatible changes so easily
> > > > > > now. It's completely out of question for a minor update (1.12.x) and must
> > > > > > be handled with extreme care in major update (1.13).
> > > > > >
> > > > > > Additionally, we already detect search bases and other config from
> > > > > > rootDSE, what makes the default SID values any different?
> > > > >
> > > > > The problem was automatic enabling of features based on rootDSE.
> > > > > User used ldap provider with AD but without idmapping. The root of problem was
> > > > > that tokengroups were enabled by default (based on rootDSE) and SIDs could not
> > > > > be downloaded. Even if they were downloaded id mapping is disabled
> > > > > It would not spped upa anything.
> > > >
> > > > It wouldn't speed up anything for the first run, true, but there is a
> > > > big potential benefit for lookups of other users.
> > > >
> > > > Consider Alice and Bob are both members of two same groups. Alice logs
> > > > in first, the cache is empty, so we look up both sids. So we have a base
> > > > search for the user, a base search for the tokenGroups and then two searches
> > > > based on SID. Then Bob comes in, we check his tokenGroups attribute and
> > > > see that both SIDs are already cached, so we just link Bob's user entry
> > > > with the SIDs and we're done -- total of two lookups with scope=base.
> > > >
> > > > So this change /does/ have a nice speed benefit, however, see below..
>
> there's nothing about POSIX lookups.

Yes, I describe the scenario with POSIX attributes enabled here. Without
POSIX attributes, initgroups is much simpler because you don't have to
look up the groups, ever. So with ID mapping, all you have to do is to
search the entry, then do a base search for TG and then derive GIDs from
the SIDs. So the number of searches when ID mapping is always constant -
two.

The optimization when it comes to POSIX attributes with tokenGroups is
that you can check what SIDs the user is a member of and only fetch
those SIDs that are not cached yet. The cached entries can be just
linked with.

The difference between using and not using TGs in this case is mainly
that without TGs you need to do several layers of searches. You'd start
with searching for member=user_dn first, that would yield some groups
which you need to recurse into. With TGs the group memberships are
unrolled already.
> > > > > We tried to solve it with this patch, but there is another solution.
> > > > > To disable tokengroups for ldap provider (rfc2307, rfc2307bis).
> > > >
> > > > I spent pretty much the whole day today triaging issues after users
> > > > upgraded to RHEL-6.6. And I tend to agree we should revert this change
> > > > for id_provider=ldap && ldap_id_mapping=False.
> > > >
> > > > Setting the SID attribute values to the AD defaults as I proposed earlier
> > > > is still something that might make sense and would solve maybe 50% of
> > > > issues. But the other 50% are strange errors like looking
> > > > up SIDs from another subdomain, where the LDAP provider has no access etc.
>
> there's nothing about POSIX lookups either.
> > > > > If someone wants to use AD as any other ldap server we should not enable ad
> > > > > features by default.
> > > >
> > > > ^^^^^^^^
> > > > I disagree, we should take advantage of whatever features we have at
> > > > our disposal. But tokenGroups along with the LDAP provider proved to be
> > > > too fragile..
> > > >
> > > > So after experiencing the breakage from downstream, I also vote for
> > > > enabling tokenGroups if:
> > > > id_provider = ad, with or without ID mapping
> > > > ldap_provider = ldap, without ID mapping
> > > >
> > > > To make sure id_provider=ldap && mapping=true we should also set the user
> > > > and group SID attribute defaults.
> > >
> > > It is clear that tokengroups and id mapping are related options.
> >
> > They are not. As I said above, tokengroups can be used to speed up POSIX
> > lookups, too.
>
> I read mails few times, but I didn't find anything about POSIX lookups.
> Did I overlook something?
>
> LS
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[REGRESSION] [PATCH] LDAP: Change defaults for ldap_user/group_objectsid

[Simo Sorce]

On Fri, 24 Oct 2014 21:14:50 +0200
Jakub Hrozek  wrote:
> On Fri, Oct 24, 2014 at 07:57:29PM +0200, Michal Židek wrote:
> > On 10/24/2014 07:09 PM, Jakub Hrozek wrote:
> > > On Fri, Oct 24, 2014 at 02:55:44PM +0200, Lukas Slebodnik wrote:
> > > > > > { "ldap_group_objectsid", NULL, SYSDB_SID, NULL },
> > > > > > { "ldap_group_objectsid", "objectSID", SYSDB_SID, NULL },
> > > > > > { "ldap_group_modify_timestamp", "modifyTimestamp",
> > > > > >
> > > > > > SYSDB_ORIG_MODSTAMP, NULL }, { "ldap_group_entry_usn", NULL,
> > > > > > SYSDB_USN, NULL }, { "ldap_group_type", NULL,
> > > > > > SYSDB_GROUP_TYPE, NULL },
> > > > >
> > > > > These changes in rfc2307_user_map and rfc2307bis_user_map caused
> > > > > regression, which was fixed by ticket
> > > > > https://fedorahosted.org/sssd/ticket/2383
> > > > >
> > > > > Dereference failed with openldapserver.
> > > > > [sdap_nested_group_process_send] (0x2000): Dereferencing members
> > > > > of group [cn=topgrp,ou=Groups,dc=example,dc=com]
> > > > > [sdap_deref_search_send] (0x2000): Server supports OpenLDAP
> > > > > deref [sdap_x_deref_search_send] (0x0400): Dereferencing entry
> > > > > [cn=topgrp,ou=Groups,dc=example,dc=com] using OpenLDAP deref
> > > > > [sdap_print_server] (0x2000): Searching 10.16.64.191
> > > > > [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext
> > > > > with [no filter][cn=topgrp,ou=Groups,dc=example,dc=com].
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs:
> > > > > [objectClass] [sdap_get_generic_ext_step] (0x1000): Requesting
> > > > > attrs: [cn] [sdap_get_generic_ext_step] (0x1000): Requesting
> > > > > attrs: [userPassword] [sdap_get_generic_ext_step] (0x1000):
> > > > > Requesting attrs: [gidNumber] [sdap_get_generic_ext_step]
> > > > > (0x1000): Requesting attrs: [member] [sdap_get_generic_ext_step]
> > > > > (0x1000): Requesting attrs: [objectSID]
> > > > > [sdap_get_generic_ext_step] (0x1000): Requesting attrs:
> > > > > [modifyTimestamp] [sdap_get_generic_ext_step] (0x1000):
> > > > > Requesting attrs: [modifyTimestamp] [sdap_get_generic_ext_step]
> > > > > (0x1000): Requesting attrs: [uid] [sdap_get_generic_ext_step]
> > > > > (0x2000): ldap_search_ext called, msgid = 3
> > > > > [sdap_process_message] (0x4000): Message type:
> > > > > [LDAP_RES_SEARCH_RESULT] [sdap_get_generic_op_finished]
> > > > > (0x0400): Search result: Protocol error(2), Dereference control:
> > > > > attribute decoding error [sdap_get_generic_op_finished]
> > > > > (0x0040): Unexpected result from ldap: Protocol error(2),
> > > > > Dereference control: attribute decoding error
> > > > > [generic_ext_search_handler] (0x0040): sdap_get_generic_ext_recv
> > > > > failed [5]: Input/output error [sdap_deref_search_done]
> > > > > (0x0040): dereference processing failed [5]: Input/output error
> > > > > [sdap_nested_group_deref_direct_done] (0x0020): Error processing
> > > > > direct membership [5]: Input/output error [sdap_nested_done]
> > > > > (0x0020): Nested group processing failed: [5][Input/output error]
> > > > >
> > > > > SSSD did equivalen of next command line search:
> > > > > [root@host ~]# ldapsearch -x -LLL -h $SERVER -b
> > > > > 'dc=example,dc=com' \
> > > > >    -E
> > > > > '!deref=member:objectClass,cn,userPassword,gidNumber,objectSID,modifyTimestamp,uid'
> > > > > \ cn=topgrp
> > > > > Protocol error (2)
> > > > > Additional information: Dereference control: attribute decoding
> > > > > error
> > > > >
> > > > > Dereferencing failed because attribute objectSID isn't in any
> > > > > objectclass on openldap server.
> > > > >
> > > > > Summary:
> > > > > We should revert this patch. ObjectSID should not be set by
> > > > > default in rfc2307 or rfc2307bis scheme.
> > > >
> > > > BTW we had the same problem with nonstandard UUID attribute
> > > > https://git.fedorahosted.org/cgit/sssd.git/commit/?id=dfb2960ab251f609466fa6...
> > > >
> > > > LS
> > >
> > > OK, this is bad, thanks a lot for catching the problem.
> > >
> > > I agree that we should revert the patch as a first step. But at
> > > the same time, the changes in 1.11 caused problems for several
> > > users running with id_provider=ldap enrolled with an AD server.
> > >
> > > In retrospective, I wish we limited some of the AD-specific
> > > features to the AD provider only, but in order to mitigate the
> > > problem, what about setting the defaults if (and only if) we
> > > detect the server is an AD server from rootDSE? We already do that
> > > for tokenGroups..
> >
> > After discussion with Lukas I think we should not do this.
> > Not even for tokenGroups. If user specifies his provider
> > as ldap, we should not do background magic that changes
> > defaults behind the scenes (based on rootDSE or anything
> > else).
>
> I totally agree, see my previous mail -- "In retrospective, I wish we
> limited some of the AD-specific features to the AD provider only".
>
> But I don't think we can do backwards-incompatible changes so easily
> now. It's completely out of question for a minor update (1.12.x) and
> must be handled with extreme care in major update (1.13).
>
> Additionally, we already detect search bases and other config from
> rootDSE, what makes the default SID values any different?

+1 to Jakub's argument.

If the LDAP server provides a capability we can use, we should use it.
We should override the LDAP server advertised capabilities only in the
negative, ie to disable them, if the client's admin know the server
misbehaves.

There is no reason to disable a feature by default if we know we can
use it.
> > Errors like the ones that my original (regression making)
> > patch tried to solve (id_mapping is set but ldap_*_objectsid
> > is not) should be detected by config file validator (once we
> > have it). Adding hacks to the code is not a good way (so
> > this patch was not good as well).
>
> Yes, but what is even worse is users running a supported and valid
> configuration and having their setup broken with a yum upgrade.

Not only that, but the code should not break if 2 conflicting options
are set. Rather it should loudly warn at level 0, and then switch one
of the 2 options to a "working" state (possibly the 'safest' one,
where "safest" is both "mostly likely to work" and "more restrictive
security-wise" if applicable).

Simo.