On (28/07/16 12:58), Lukas Slebodnik wrote:
On (28/07/16 12:44), thierry bordaz wrote:
>
>
>On 07/28/2016 11:17 AM, Lukas Slebodnik wrote:
>> On (28/07/16 10:24), thierry bordaz wrote:
>> >
>> > On 07/28/2016 09:39 AM, Jakub Hrozek wrote:
>> > > On Wed, Jul 27, 2016 at 04:09:07PM +0200, thierry bordaz wrote:
>> > > > On 07/27/2016 03:36 PM, Jakub Hrozek wrote:
>> > > > > On Wed, Jul 27, 2016 at 02:55:37PM +0200, thierry bordaz
wrote:
>> > > > > > On 07/27/2016 01:56 PM, Jakub Hrozek wrote:
>> > > > > > > On Wed, Jul 27, 2016 at 01:03:59PM +0200, Jakub
Hrozek wrote:
>> > > > > > > > On Wed, Jul 27, 2016 at 12:22:46PM +0200,
Lukas Slebodnik wrote:
>> > > > > > > > > On (27/07/16 12:08), Jakub Hrozek wrote:
>> > > > > > > > > > On Wed, Jul 27, 2016 at 12:02:24PM
+0200, Jakub Hrozek wrote:
>> > > > > > > > > > > On Wed, Jul 27, 2016 at
11:54:16AM +0200, Lukas Slebodnik wrote:
>> > > > > > > > > > > > ehlo,
>> > > > > > > > > > > >
>> > > > > > > > > > > > attached patch fixes acces
denied after activating user in 389ds.
>> > > > > > > > > > > > Jakub had some
comments/ideas in ticket but I think it's better to discuss
>> > > > > > > > > > > > about virtual attributes
and timestamp cache on mailing list.
>> > > > > > > > > > > Yes, so the comment I have is
that while this works, it might break some
>> > > > > > > > > > > strange LDAP servers.
>> > > > > > > > > > >
>> > > > > > > > > > > We use modifyTimestamp as a
'positive' indicator that the entry has not
>> > > > > > > > > > > changed -- if the
modifyTimestamp didn't change, we consider the cached
>> > > > > > > > > > > entry the same as what is on
the server and only bump the timestamp
>> > > > > > > > > > > cache. If the timestamp is
different, we do a deep-comparison of cached
>> > > > > > > > > > > attribute values with what is
on the LDAP server and write the sysdb
>> > > > > > > > > > > cache entry only if the
attributes differ.
>> > > > > > > > > > >
>> > > > > > > > > > > I was wondering if we can use
the modifyTimestamp at all, then, because
>> > > > > > > > > > > even if it's the same, we
might want to check the attributes to see if
>> > > > > > > > > > > some of the values are
different because some of the attributes might be
>> > > > > > > > > > > this operational/virtual
attribute..
>> > > > > > > > > > Sorry, sent too soon.
>> > > > > > > > > >
>> > > > > > > > > > I think the questions are -- 1) can
we enumerate the virtual attributes?
>> > > > > > > > > That might be a question for 389-ds
developers.
>> > > > > > > > > But it's very likely it will be
different on other LDAP servers.
>> > > > > > > > >
>> > > > > > > > > > 2) Would different LDAP servers have
different virtual attributes.
>> > > > > > > > > >
>> > > > > > > > > > For 2) maybe a possible solution
might be to set a non-existing
>> > > > > > > > > > modifyTimestamp attribute value, but
I would consider that only a
>> > > > > > > > > > kludge, we shouldn't break
existing setups..
>> > > > > > > > > I am not satisfied with this POC solution
either.
>> > > > > > > > >
>> > > > > > > > > So should we remove usage of
modifyTimestamp for detecting changes?
>> > > > > > > > I would prefer to ask the DS developers before
removing it completely.
>> > > > > > > >
>> > > > > > > > At least for large groups it might take a long
time to compare all attribute
>> > > > > > > > values and IIRC we don't depend on any
virtual attributes for groups. Maybe
>> > > > > > > > we could parametrize that part of the code and
enable the fast way with
>> > > > > > > > modifyTimestamps for 'known' server
types, that is for setups with AD and
>> > > > > > > > IPA providers.
>> > > > > > > >
>> > > > > > > > For users, there is typically not as many
attributes so we should be
>> > > > > > > > fine deep-comparing all attributes.
>> > > > > > > I'm adding Thierry (so please reply-to-all to
keep him in the thread).
>> > > > > > >
>> > > > > > > Thierry, in the latest sssd version we tried to add
a performance
>> > > > > > > improvement related to how we store SSSD entries in
the cache. The short
>> > > > > > > version is that we store the modifyTimestamp
attribute in the cache and
>> > > > > > > when we fetch an entry, we compare the entry
modifyTimestamp with what
>> > > > > > > is on the server. When the two are the same, we say
that the entry did
>> > > > > > > not change and don't update the cache.
>> > > > > > >
>> > > > > > > This works fine for most attributes, but not for
attributes like
>> > > > > > > nsAccountLock which do not change modifyTimestamp
when they are
>> > > > > > > modified. So when an entry was already cached but
then nsAccountLock
>> > > > > > > changed, we treated the entry as the same and never
read the new
>> > > > > > > nsAccountLock value.
>> > > > > > >
>> > > > > > > To fix this, I think we have several options:
>> > > > > > > 1) special-case the nsAccountLock. This
seems a bit dangerous,
>> > > > > > > because I'm not sure we can say that
some other attribute we are
>> > > > > > > interested in behaves the same as
nsAccountLock.
>> > > > > > > 2) drop the modifyTimestamp optimization
completely. Then we fall
>> > > > > > > back to comparing the attribute values,
which might work, but for
>> > > > > > > huge objects like groups with thousands of
members, this might be
>> > > > > > > too expensive.
>> > > > > > > 3) only use the modifyTimestamp
optimization for cases where we know
>> > > > > > > we don't read any virtual attributes.
>> > > > > > >
>> > > > > > > And my question is -- can we, in general, know if
the modifyTimestamp
>> > > > > > > way of detecting changes is realiable for all LDAP
servers? Or do you
>> > > > > > > think it should only be used for cases where we
know we are not
>> > > > > > > interested in any virtual attributes (that would
mostly be storing
>> > > > > > > groups from servers where we know exactly what is
on the server side,
>> > > > > > > like IPA or AD).
>> > > > > > Hello,
>> > > > > >
>> > > > > > Relying on modifytimestamp looks a good idea. Any
MOD/MODRDN will update it,
>> > > > > > except I think it is unchanged when updating some
password policy
>> > > > > > attributes.
>> > > > > >
>> > > > > > Regarding virtual attribute, the only one I know in IPA
is nsaccountlock.
>> > > > > > nsaccountlock is an operational attribute (you need to
request it to see it)
>> > > > > > and is also a virtual attribute BUT only for
'staged' and 'deleted' users.
>> > > > > > It is a stored attribute for regular users and we should
update
>> > > > > > modifytimestamp when it is set.
>> > > > > >
>> > > > > > thanks
>> > > > > > thierry
>> > > > > OK, in that case it seems like we can special-case it. But do
you know
>> > > > > about any other attributes in any other LDAP servers?
>> > > > Any LDAP server following standard should provide modifytimestamp
that
>> > > > reflect the last update of the entry. Now virtual attribute values
may be
>> > > > "attached" to the entry and its value change without
modification of
>> > > > modifytimestamp.
>> > > > For 389-ds and IPA it is fine as virtual value of nsaccountlock is
changed
>> > > > only when the DN change.
>> > > > For others LDAP servers I suppose it exists the same ability to
define
>> > > > service providers that return virtual attribute values. The
difficulty is
>> > > > that the schema may not give any hint if the retrieved attributes
values
>> > > > were stored or computed and consequently trust modifytimestamp to
know if
>> > > > the values changed or not.
>> > > > For example in ODSEE, memberof is a virtual attribute.
>> > > Thank you, for the explanation Thierry.
>> > >
>> > > Then to be on the safe side I propose:
>> > > 1) We add an (probably undocumented?) flag that says whether to
use
>> > > modifyTimestamp to detect entry changes or not
>> > > 2) for the generic LDAP provider we always really compare the
>> > > attribute values, in other words the option would be set to
>> > > false. If there is anyone with performance issues with a
generic
>> > > setup, we tell them to flip the option.
>> > > 3) For the IPA and AD providers, we set this option to true and
use
>> > > the modifyTimestamp value to detect changes
>> > > 4) We special case nsAccountLock
>> > I am not sure I understand why nsAccountLock is a special case ?
>> > In IPA/389:
>> >
>> > * Stage user, it is always 'nsaccountlock: True'
>> > When stage entry is created or updated, 'modifytimestamp' is
also
>> > updated. So you can rely on modifytimestamp to detect a change in a
>> > stage user.
>> > There is no way to update nsaccountlock for a stage entry
>> > * Deleted user. Idem Stage user
>> We haven't tested stage users yet. I ran just sssd related regression tests.
>>
>> > * Regular user. 'nsaccountlock' is *not* a virtual attribute, so if
it
>> > is enable/disable you can rely on modifytimestamp to detect a change
>> > of 'nsaccountlock' for a regular user.
>> > Also any change on regular user will update 'modifytimestamp' so
you
>> > can rely on it to detect a change.
>> >
>> My experience with ns-inactivate.pl and ns-activate.pl is different.
>> modifytimestamp is not changed even though nsaccountlock was changed.
>>
>> It was a plain 389-ds with id_provider ldap in sssd
>>
>> LS
>Hi Lukas,
>
>
>It may have change recently because I can not reproduce. What versions are
>you running ?
>
> #
> #initial entry
> #
> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b
> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> createtimestamp
> dn: uid=user0,cn=users,cn=accounts,<suffix>
> modifytimestamp: *20160728103441Z*
> createtimestamp: 20160726160436Z
>
> #
> # Inactivate: modifytimestamp changed
> #
> /usr/sbin/ns-inactivate.pl -Z <instance> -D "cn=directory manager"
> -w xxx -I uid=user0,cn=users,cn=accounts,<suffix>
> uid=user0,cn=users,cn=accounts,<suffix> inactivated.
>
> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b
> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> createtimestamp
> dn: uid=user0,cn=users,cn=accounts,<suffix>
> nsaccountlock: true
> modifytimestamp: *20160728103642Z*
> createtimestamp: 20160726160436Z
>
> #
> # activate: modifytimestamp changed
> #
> /usr/sbin/ns-activate.pl -Z <instance> -D "cn=directory manager"
-w
> xxx -I uid=user0,cn=users,cn=accounts,<suffix>
> uid=user0,cn=users,cn=accounts,<suffix> activated.
>
> ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx -b
> "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> createtimestamp
> dn: uid=user0,cn=users,cn=accounts,<suffix>
> modifytimestamp: *20160728103711Z*
> createtimestamp: 20160726160436Z
>
rhel7.3
389-ds-base-1.3.5.10-5.el7.x86_64
But I didn't test directly with user but indirectly
*Add Managed role
dn: cn=managed,ou=people,dc=example,dc=com
objectclass: top
objectclass: LdapSubEntry
objectclass: nsRoleDefinition
objectclass: nsSimpleRoleDefinition
objectclass: nsManagedRoleDefinition
cn: Managed
*Authenticate with lockuser
*Add user to managed role
dn: uid=lockuser,ou=people,dc=example,dc=com
changetype: modify
add: nsRoleDN
nsRoleDN: cn=managed,ou=people,dc=example,dc=com
ns-inactivate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \
-h $SERVER -I cn=managed,ou=people,dc=example,dc=com
*Authenticate with lockuser
ns-activate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \
-h $SERVER -I cn=managed,ou=people,dc=example,dc=com
*Authenticate with lockuser
Should I test with rhel7.2 or is it an expected behaviour ?
I tested with rhel7.2 389-ds and there is the same behaviour.
The attribute modifytimestamp was not changed for user.
LS