On (28/07/16 10:24), thierry bordaz wrote:
On 07/28/2016 09:39 AM, Jakub Hrozek wrote:
> On Wed, Jul 27, 2016 at 04:09:07PM +0200, thierry bordaz wrote:
> >
> > On 07/27/2016 03:36 PM, Jakub Hrozek wrote:
> > > On Wed, Jul 27, 2016 at 02:55:37PM +0200, thierry bordaz wrote:
> > > > On 07/27/2016 01:56 PM, Jakub Hrozek wrote:
> > > > > On Wed, Jul 27, 2016 at 01:03:59PM +0200, Jakub Hrozek wrote:
> > > > > > On Wed, Jul 27, 2016 at 12:22:46PM +0200, Lukas Slebodnik
wrote:
> > > > > > > On (27/07/16 12:08), Jakub Hrozek wrote:
> > > > > > > > On Wed, Jul 27, 2016 at 12:02:24PM +0200, Jakub
Hrozek wrote:
> > > > > > > > > On Wed, Jul 27, 2016 at 11:54:16AM +0200,
Lukas Slebodnik wrote:
> > > > > > > > > > ehlo,
> > > > > > > > > >
> > > > > > > > > > attached patch fixes acces denied after
activating user in 389ds.
> > > > > > > > > > Jakub had some comments/ideas in ticket
but I think it's better to discuss
> > > > > > > > > > about virtual attributes and timestamp
cache on mailing list.
> > > > > > > > > Yes, so the comment I have is that while this
works, it might break some
> > > > > > > > > strange LDAP servers.
> > > > > > > > >
> > > > > > > > > We use modifyTimestamp as a
'positive' indicator that the entry has not
> > > > > > > > > changed -- if the modifyTimestamp didn't
change, we consider the cached
> > > > > > > > > entry the same as what is on the server and
only bump the timestamp
> > > > > > > > > cache. If the timestamp is different, we do a
deep-comparison of cached
> > > > > > > > > attribute values with what is on the LDAP
server and write the sysdb
> > > > > > > > > cache entry only if the attributes differ.
> > > > > > > > >
> > > > > > > > > I was wondering if we can use the
modifyTimestamp at all, then, because
> > > > > > > > > even if it's the same, we might want to
check the attributes to see if
> > > > > > > > > some of the values are different because some
of the attributes might be
> > > > > > > > > this operational/virtual attribute..
> > > > > > > > Sorry, sent too soon.
> > > > > > > >
> > > > > > > > I think the questions are -- 1) can we enumerate
the virtual attributes?
> > > > > > > That might be a question for 389-ds developers.
> > > > > > > But it's very likely it will be different on other
LDAP servers.
> > > > > > >
> > > > > > > > 2) Would different LDAP servers have different
virtual attributes.
> > > > > > > >
> > > > > > > > For 2) maybe a possible solution might be to set a
non-existing
> > > > > > > > modifyTimestamp attribute value, but I would
consider that only a
> > > > > > > > kludge, we shouldn't break existing setups..
> > > > > > > I am not satisfied with this POC solution either.
> > > > > > >
> > > > > > > So should we remove usage of modifyTimestamp for
detecting changes?
> > > > > > I would prefer to ask the DS developers before removing it
completely.
> > > > > >
> > > > > > At least for large groups it might take a long time to
compare all attribute
> > > > > > values and IIRC we don't depend on any virtual
attributes for groups. Maybe
> > > > > > we could parametrize that part of the code and enable the
fast way with
> > > > > > modifyTimestamps for 'known' server types, that is
for setups with AD and
> > > > > > IPA providers.
> > > > > >
> > > > > > For users, there is typically not as many attributes so we
should be
> > > > > > fine deep-comparing all attributes.
> > > > > I'm adding Thierry (so please reply-to-all to keep him in the
thread).
> > > > >
> > > > > Thierry, in the latest sssd version we tried to add a
performance
> > > > > improvement related to how we store SSSD entries in the cache.
The short
> > > > > version is that we store the modifyTimestamp attribute in the
cache and
> > > > > when we fetch an entry, we compare the entry modifyTimestamp with
what
> > > > > is on the server. When the two are the same, we say that the
entry did
> > > > > not change and don't update the cache.
> > > > >
> > > > > This works fine for most attributes, but not for attributes like
> > > > > nsAccountLock which do not change modifyTimestamp when they are
> > > > > modified. So when an entry was already cached but then
nsAccountLock
> > > > > changed, we treated the entry as the same and never read the new
> > > > > nsAccountLock value.
> > > > >
> > > > > To fix this, I think we have several options:
> > > > > 1) special-case the nsAccountLock. This seems a bit
dangerous,
> > > > > because I'm not sure we can say that some other
attribute we are
> > > > > interested in behaves the same as nsAccountLock.
> > > > > 2) drop the modifyTimestamp optimization completely. Then
we fall
> > > > > back to comparing the attribute values, which might work,
but for
> > > > > huge objects like groups with thousands of members, this
might be
> > > > > too expensive.
> > > > > 3) only use the modifyTimestamp optimization for cases
where we know
> > > > > we don't read any virtual attributes.
> > > > >
> > > > > And my question is -- can we, in general, know if the
modifyTimestamp
> > > > > way of detecting changes is realiable for all LDAP servers? Or do
you
> > > > > think it should only be used for cases where we know we are not
> > > > > interested in any virtual attributes (that would mostly be
storing
> > > > > groups from servers where we know exactly what is on the server
side,
> > > > > like IPA or AD).
> > > > Hello,
> > > >
> > > > Relying on modifytimestamp looks a good idea. Any MOD/MODRDN will
update it,
> > > > except I think it is unchanged when updating some password policy
> > > > attributes.
> > > >
> > > > Regarding virtual attribute, the only one I know in IPA is
nsaccountlock.
> > > > nsaccountlock is an operational attribute (you need to request it to
see it)
> > > > and is also a virtual attribute BUT only for 'staged' and
'deleted' users.
> > > > It is a stored attribute for regular users and we should update
> > > > modifytimestamp when it is set.
> > > >
> > > > thanks
> > > > thierry
> > > OK, in that case it seems like we can special-case it. But do you know
> > > about any other attributes in any other LDAP servers?
> > Any LDAP server following standard should provide modifytimestamp that
> > reflect the last update of the entry. Now virtual attribute values may be
> > "attached" to the entry and its value change without modification of
> > modifytimestamp.
> > For 389-ds and IPA it is fine as virtual value of nsaccountlock is changed
> > only when the DN change.
> > For others LDAP servers I suppose it exists the same ability to define
> > service providers that return virtual attribute values. The difficulty is
> > that the schema may not give any hint if the retrieved attributes values
> > were stored or computed and consequently trust modifytimestamp to know if
> > the values changed or not.
> > For example in ODSEE, memberof is a virtual attribute.
> Thank you, for the explanation Thierry.
>
> Then to be on the safe side I propose:
> 1) We add an (probably undocumented?) flag that says whether to use
> modifyTimestamp to detect entry changes or not
> 2) for the generic LDAP provider we always really compare the
> attribute values, in other words the option would be set to
> false. If there is anyone with performance issues with a generic
> setup, we tell them to flip the option.
> 3) For the IPA and AD providers, we set this option to true and use
> the modifyTimestamp value to detect changes
> 4) We special case nsAccountLock
I am not sure I understand why nsAccountLock is a special case ?
In IPA/389:
* Stage user, it is always 'nsaccountlock: True'
When stage entry is created or updated, 'modifytimestamp' is also
updated. So you can rely on modifytimestamp to detect a change in a
stage user.
There is no way to update nsaccountlock for a stage entry
* Deleted user. Idem Stage user
We haven't tested stage users yet. I ran just
sssd related regression tests.
* Regular user. 'nsaccountlock' is *not* a virtual attribute,
so if it
is enable/disable you can rely on modifytimestamp to detect a change
of 'nsaccountlock' for a regular user.
Also any change on regular user will update 'modifytimestamp' so you
can rely on it to detect a change.
My experience with ns-inactivate.pl and ns-activate.pl is different.
modifytimestamp is not changed even though nsaccountlock was changed.
It was a plain 389-ds with id_provider ldap in sssd
LS