On Thu, 2009-08-13 at 08:54 -0400, Stephen Gallagher wrote:
One potential idea would be to have the SSSD automatically start an
enumeration at startup time if the cache is stale. Then, instead of
blocking updates waiting for subsequent enumerations, we could just go
immediately to the cache until the enumeration was complete.
See below, this is what I propose :)
> After some hard thinking I wrote down a few points I'd like
> opinion on. If people agree I will start acting on them.
> * stop performing enumerations on demand, and perform them in background
> if enumerations are activated (change the enumeration parameter from a
> bitfield to a true/flase boolean)
> * perform a full user+group enumeration at startup (possibly using a
> paged or vlv search)
> * when possible request the modifyTimestamp attribute and save the
> highest modifyTimestamp into the domain entry as originalMaxTimestamp
> * on a tunable interval run a task that refreshes all users and groups
> in the background using a search filter that includes
> * still do a full refresh every X minutes/hours
> * disable using a single huge transaction for enumerations (we might be
> ok doing a transaction for each page search if pages are small,
> otherwise just revert to the previous behavior of having a transaction
> per stored object)
> * Every time we update an entry we store the originalModifyTimestamp on
> it as a copy of the remote modifyTimestamp, this allows us to know if we
> actually need to touch the cached entry at all upon refresh (like when a
> getpwnam() is called, speeding up operations for entries that need no
> refresh (we will avoid data transformation and a writing to ldb).
> * Every time we run the general refresh task or we save a changed entry
> we store a LastUpdatedTimestamp
> * When the refresh task is completed successfully we run another cleanup
> task that searches our LDB for any entry that has a too old
> LastUpdatedTimestamp. If any is found, we double check against the
> remote server if the entry still exists (and update it if it does), and
> otherwise we delete it.
I do like the idea of updating only the entries on the remote server
that have been updated since the previous enumeration.
Yes hopefully all servers allow this kind of query, I guess we need to
experiment and have options to fallback to full searches if this is not
> NOTE: this means that until the first background enumeration is
> complete, a getent passwd or a getent group call may return incomplete
> results. I think this is acceptable as it will really happen only at
> startup, when the daemon caches are empty.
I disagree. If we're going to have a startup enumeration, then we should
simply not enable handling NSS requests until that first enumeration is
complete. Incomplete results can be worse than no results. I assume NSS
has a return code for temporary failure?
Internally, yes, but all it does it to return no results to the user
space. Not returning results is == returning partial results. So I see
no difference here.
Note that this will happen only on the first startup when caches are
empty as we otherwise always return whatever is in the cache.