Developing custom SSSD data provider backends

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

For the last week, I've been working on developing a set of guidelines
on how to create a custom SSSD data provider backend. These backends
would be capable of performing identification and/or authentication to
custom mechanisms (such as winbind, NIS, SecurID, etc.)

First, I'd like to describe the way things work internally (for the
benefit of anyone not intimately familiar), then I'll go into the issues
we're facing with allowing custom backends.

Currently, the SSSD provides two services that face the system: NSS and
PAM. This is done by way of very lightweight NSS and PAM modules that
can be loaded automatically by nsswitch.conf or the PAM stack and that
then pass the request across a privileged pipe to the sssd_nss or
sssd_pam services (as appropriate).

Each time a request comes in, the service first queries the internal
sysdb to determine whether the entry already exists in an unexpired
cache entry. If it does not exist (or is expired), the service then must
communicate directly with a backend provider to update the cache. This
is done over a private pipe using the D-BUS protocol (we refer to
communications over this pipe as SBUS communications, so as not to be
confused with communications going over the system bus).

When a data provider backend receives the request, it then contacts its
own information source (LDAP, NIS, etc.) in an appropriate manner,
retrieving the information and updating the sysdb cache. It then returns
a message to the NSS/PAM service stating only whether the communication
succeeded (and if not, an error message describing the problem).

Assuming that it succeeded, the service will then query the sysdb again,
this time receiving an updated entry.

Now, it is important to note here that at no time does the communication
on the SBUS carry the result of the lookup that is performed by the
provider backend. It only carries the success/failure message. The true
final result is only returned from the sysdb, when it is queried after
the provider replies.

Because of this design, it is necessary for all data providers to be
capable of writing directly to the sysdb. This introduces some serious
concerns for potential backend implementers. The sysdb is designed to be
an asynchronous interface using the tevent mainloop. There is currently
no way to integrate the sysdb with any other mainloop, because at its
heart, the sysdb operates on an LDB database, which also mandates the
use of tevent for asynchronous operation.

So, in order to implement a backend provider, the provider must be
written in C and built around libtevent. This is not likely to be an
acceptable set of requirements for many implementers.

I've been trying to work around this, and I have come up with a few
potential approaches to this problem.

1) We can create a sync wrapper around all calls to the sysdb and hides
the requirement for tevent. Internally, all calls to the actual LDB are
effectively synchronous, so the performance shouldn't be too poor,
though there is still a chance that some of the longer operations (like
initgroups commands with large user bases) may block for an extended
period of time.

2) We can fork a process/spawn a thread to do the processing outside of
main execution and signal our completion. As I understand it, this is
the approach SELinux takes to asynchronous operation. This would likely
be a tremendous amount of work.

3) We can create a new SSSD service (sssd_sysdb?) that exposes the sysdb
over the SBUS(or even the system bus) restricted to root processes. This
might be the most future-compatible approach, but it too is a lot of
work (though I believe it to be less work than option 2)

4) We can go back to our original plan of having third-party
implementers follow our internal example and write shared objects that
can be loaded by the sssd_be process. This approach handles the mainloop
integration internally, thereby simplifying it for the implementers, but
it carries potential licensing issues. Since the sssd_be is licensed as
GPL, any shared object that it loads needs to be GPL-compatible. This
won't fly for companies that want to create a closed-source backend.
This is the least amount of work (almost none).

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkt0Rr4ACgkQeiVVYja6o6PjbACfUFDIqTNHDiduguFv+lrIGbrZ
> LgwAoKPmPp6GZYsydREqPz9/J2EhKVhX
> =nmc3
> -----END PGP SIGNATURE-----

[Dmitri Pal]

Stephen Gallagher wrote:
>  For the last week, I've been working on developing a set of
> guidelines
>  on how to create a custom SSSD data provider backend. These backends
>  would be capable of performing identification and/or authentication to
>  custom mechanisms (such as winbind, NIS, SecurID, etc.)
>
>  First, I'd like to describe the way things work internally (for the
>  benefit of anyone not intimately familiar), then I'll go into the issues
>  we're facing with allowing custom backends.
>
>  Currently, the SSSD provides two services that face the system: NSS and
>  PAM. This is done by way of very lightweight NSS and PAM modules that
>  can be loaded automatically by nsswitch.conf or the PAM stack and that
>  then pass the request across a privileged pipe to the sssd_nss or
>  sssd_pam services (as appropriate).
>
>  Each time a request comes in, the service first queries the internal
>  sysdb to determine whether the entry already exists in an unexpired
>  cache entry. If it does not exist (or is expired), the service then must
>  communicate directly with a backend provider to update the cache. This
>  is done over a private pipe using the D-BUS protocol (we refer to
>  communications over this pipe as SBUS communications, so as not to be
>  confused with communications going over the system bus).
>
>  When a data provider backend receives the request, it then contacts its
>  own information source (LDAP, NIS, etc.) in an appropriate manner,
>  retrieving the information and updating the sysdb cache. It then returns
>  a message to the NSS/PAM service stating only whether the communication
>  succeeded (and if not, an error message describing the problem).
>
>  Assuming that it succeeded, the service will then query the sysdb again,
>  this time receiving an updated entry.
>
>  Now, it is important to note here that at no time does the communication
>  on the SBUS carry the result of the lookup that is performed by the
>  provider backend. It only carries the success/failure message. The true
>  final result is only returned from the sysdb, when it is queried after
>  the provider replies.
>
>  Because of this design, it is necessary for all data providers to be
>  capable of writing directly to the sysdb. This introduces some serious
>  concerns for potential backend implementers. The sysdb is designed to be
>  an asynchronous interface using the tevent mainloop. There is currently
>  no way to integrate the sysdb with any other mainloop, because at its
>  heart, the sysdb operates on an LDB database, which also mandates the
>  use of tevent for asynchronous operation.
>
>  So, in order to implement a backend provider, the provider must be
>  written in C and built around libtevent. This is not likely to be an
>  acceptable set of requirements for many implementers.
>
>  I've been trying to work around this, and I have come up with a few
>  potential approaches to this problem.
>
>  1) We can create a sync wrapper around all calls to the sysdb and hides
>  the requirement for tevent. Internally, all calls to the actual LDB are
>  effectively synchronous, so the performance shouldn't be too poor,
>  though there is still a chance that some of the longer operations (like
>  initgroups commands with large user bases) may block for an extended
>  period of time.
>
>  2) We can fork a process/spawn a thread to do the processing outside of
>  main execution and signal our completion. As I understand it, this is
>  the approach SELinux takes to asynchronous operation. This would likely
>  be a tremendous amount of work.
>
>  3) We can create a new SSSD service (sssd_sysdb?) that exposes the sysdb
>  over the SBUS(or even the system bus) restricted to root processes. This
>  might be the most future-compatible approach, but it too is a lot of
>  work (though I believe it to be less work than option 2)
>
>  4) We can go back to our original plan of having third-party
>  implementers follow our internal example and write shared objects that
>  can be loaded by the sssd_be process. This approach handles the mainloop
>  integration internally, thereby simplifying it for the implementers, but
>  it carries potential licensing issues. Since the sssd_be is licensed as
>  GPL, any shared object that it loads needs to be GPL-compatible. This
>  won't fly for companies that want to create a closed-source backend.
>  This is the least amount of work (almost none).
>  
In case of option 4) we can probably suggest closed source customers to
create a thin library that will implement a predefined interface and
will marshal and unmarshal the request to a separate proprietary daemon.
If we then provide the skeleton of such library and daemon as a sample
code with explicit comment: "can be used in proprietary software" we
might be all set. What do you think?


_______________________________________________
sssd-devel mailing list
sssd-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/sssd-devel

-- 
Thank you,
Dmitri Pal

Engineering Manager IPA project,
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/11/2010 01:41 PM, Dmitri Pal wrote:
>  Stephen Gallagher wrote:
> > For the last week, I've been working on developing a set of guidelines
> > on how to create a custom SSSD data provider backend. These backends
> > would be capable of performing identification and/or authentication to
> > custom mechanisms (such as winbind, NIS, SecurID, etc.)
> >
> > First, I'd like to describe the way things work internally (for the
> > benefit of anyone not intimately familiar), then I'll go into the issues
> > we're facing with allowing custom backends.
> >
> > Currently, the SSSD provides two services that face the system: NSS and
> > PAM. This is done by way of very lightweight NSS and PAM modules that
> > can be loaded automatically by nsswitch.conf or the PAM stack and that
> > then pass the request across a privileged pipe to the sssd_nss or
> > sssd_pam services (as appropriate).
> >
> > Each time a request comes in, the service first queries the internal
> > sysdb to determine whether the entry already exists in an unexpired
> > cache entry. If it does not exist (or is expired), the service then must
> > communicate directly with a backend provider to update the cache. This
> > is done over a private pipe using the D-BUS protocol (we refer to
> > communications over this pipe as SBUS communications, so as not to be
> > confused with communications going over the system bus).
> >
> > When a data provider backend receives the request, it then contacts its
> > own information source (LDAP, NIS, etc.) in an appropriate manner,
> > retrieving the information and updating the sysdb cache. It then returns
> > a message to the NSS/PAM service stating only whether the communication
> > succeeded (and if not, an error message describing the problem).
> >
> > Assuming that it succeeded, the service will then query the sysdb again,
> > this time receiving an updated entry.
> >
> > Now, it is important to note here that at no time does the communication
> > on the SBUS carry the result of the lookup that is performed by the
> > provider backend. It only carries the success/failure message. The true
> > final result is only returned from the sysdb, when it is queried after
> > the provider replies.
> >
> > Because of this design, it is necessary for all data providers to be
> > capable of writing directly to the sysdb. This introduces some serious
> > concerns for potential backend implementers. The sysdb is designed to be
> > an asynchronous interface using the tevent mainloop. There is currently
> > no way to integrate the sysdb with any other mainloop, because at its
> > heart, the sysdb operates on an LDB database, which also mandates the
> > use of tevent for asynchronous operation.
> >
> > So, in order to implement a backend provider, the provider must be
> > written in C and built around libtevent. This is not likely to be an
> > acceptable set of requirements for many implementers.
> >
> > I've been trying to work around this, and I have come up with a few
> > potential approaches to this problem.
> >
> > 1) We can create a sync wrapper around all calls to the sysdb and hides
> > the requirement for tevent. Internally, all calls to the actual LDB are
> > effectively synchronous, so the performance shouldn't be too poor,
> > though there is still a chance that some of the longer operations (like
> > initgroups commands with large user bases) may block for an extended
> > period of time.
> >
> > 2) We can fork a process/spawn a thread to do the processing outside of
> > main execution and signal our completion. As I understand it, this is
> > the approach SELinux takes to asynchronous operation. This would likely
> > be a tremendous amount of work.
> >
> > 3) We can create a new SSSD service (sssd_sysdb?) that exposes the sysdb
> > over the SBUS(or even the system bus) restricted to root processes. This
> > might be the most future-compatible approach, but it too is a lot of
> > work (though I believe it to be less work than option 2)
> >
> > 4) We can go back to our original plan of having third-party
> > implementers follow our internal example and write shared objects that
> > can be loaded by the sssd_be process. This approach handles the mainloop
> > integration internally, thereby simplifying it for the implementers, but
> > it carries potential licensing issues. Since the sssd_be is licensed as
> > GPL, any shared object that it loads needs to be GPL-compatible. This
> > won't fly for companies that want to create a closed-source backend.
> > This is the least amount of work (almost none).
> >
>  In case of option 4) we can probably suggest closed source customers to
>  create a thin library that will implement a predefined interface and
>  will marshal and unmarshal the request to a separate proprietary daemon.
>  If we then provide the skeleton of such library and daemon as a sample
>  code with explicit comment: "can be used in proprietary software" we
>  might be all set. What do you think? 

I don't think we can reasonably do this. There are quite a few sysdb
interfaces that the customer would be required to write to. I'm not sure
that we could create a sufficiently generic example library.

If that's the route we wanted to go, it would be wiser to implement
option 3)

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkt0Uz4ACgkQeiVVYja6o6NC4wCdERY9hHW6eEV9NkEgS6gjgq/b
> MjkAniIVdH5OZWOmGRzni/2UkyrUgHr4
> =Tq34
> -----END PGP SIGNATURE-----

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/11/2010 01:58 PM, Stephen Gallagher wrote:
> >> 3) We can create a new SSSD service (sssd_sysdb?) that
> exposes the sysdb
> >> over the SBUS(or even the system bus) restricted to root processes. This
> >> might be the most future-compatible approach, but it too is a lot of
> >> work (though I believe it to be less work than option 2) 
After a long discussion of the pros and cons off-list, we're going to
take this approach.

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkt0aUkACgkQeiVVYja6o6MNrwCfZFmols0YApY0rYnIhK47h46w
> lbYAn00l/T3k7eKI+dq3/ZgaGSo9w7wB
> =fYf1
> -----END PGP SIGNATURE-----