On Oct 5, 2011, at 4:30 AM, Stephen Gallagher wrote:
CCing sssd-devel
On Tue, 2011-10-04 at 22:41 +0000, JR Aquino wrote:
> So I have been looking through the results of ldbsearch and have been
> wondering...
>
> Why is it that I see cached entries for objects (hbac/ hosts/ etc) that
> have no correlation to the host itself?
>
> Might it be more optimized to search / cache only the data relevant to
> the client system in question?
>
> Surely that would cut down on the exponential costs against an ldap
> database with grown hosts/users/hbacrules/sudorules ?
Well, the issue is this: rules can specify hostgroups as well as hosts.
So in order to decide which rules to download, we need to know which
hostgroups the current machine is a member of. This results in a minimum
of two ldap searches (one for the hosts, and then one for the
hostgroups) to get the machine-specific information. We then need to
perform a rule search for just the rules that apply to that machine.
After we have the rule, we'll need to perform an iterative series of
lookups for every host and hostgroup that the rules we just acquired
list as a source host. Now, suppose that one or more of the hostgroups
specified in source host contains thousands of host entries?
Can we not just perform a targeted search of the Originating Source's hostname to
query the hostgroups that it is a memberof and if it matches any of the hostgroups listed
as a srchost = pass/fail the hbac?
While there is the potential that 'every' host in the ldap directory could
possibly attempt to login to a system, the likely hood seems far less probable vs caching
only those systems / users who come calling?
Won't there be a measurable exponential graph to follow as the contents of the ldap
directory grows, so too does the process intensity of the sssd caching/processing vs only
querying and caching he who knocks?
So in other words, doing it this way will result in a net increase in
the number of back-and-forth LDAP exchanges. In theory this is much
slower than just grabbing everything in two lookups up front (getting
all hosts and hostgroups) and doing the processing locally.
In practice, we're hitting a performance issue because of the way we're
updating the data. So it's better all-around to resolve the issue here,
rather than deferring the problem by changing the lookup order around
and potentially increasing the LDAP lookups. And since we still might
end up with rules that have to handle a lot of hosts, we'll still have
the same update issue.
The problem in the source is this: I tried to simplify the update
process by simply wiping out the hosts and hostgroups subtrees in the
cache and then writing the new values in. The problem with this is that
deleting entries with memberOf attributes is an expensive operation,
which I did not take into account.
There are two approaches to resolving this that I'm taking into
consideration.
1) Instead of doing a full purge followed by a full write, make only
changes. This will be a somewhat more complex logic to follow, but if
little or nothing has changed on the server, then it will save us a LOT
of memberOf operations.
2) Add a new feature to the memberOf plugin to allow us to defer
processing memberOf until all entries have first been written (keeping
an index of the attributes that will need processing). This is *very*
complicated and potentially dangerous (though also has the potential for
the highest performance gain), so I'd prefer to try option 1 first.