Design Discussion: IPA Server Mode

[Sumit Bose]

Hi,

I have created a design page for one of the next major features of SSSD
at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
basic idea is that if SSSD is running on a FreeIPA server it should help
the FreeIPA server to look up users and groups from trusted domains.

For your convenience the content can be found below as well.

Comments are suggestions are welcome.

bye,
Sumit

== IPA Server Mode ==
Related tickets:
* [https://fedorahosted.org/sssd/ticket/1821 RFE Allow using UIDs and GIDs from AD in trust case]
* [https://fedorahosted.org/sssd/ticket/1881 RFE Determine how to map SID to UID/GID based on IdM server configuration]
* more to come

=== Problem Statement ===
FreeIPA is planning to make users and groups from trusted domains available to legacy systems, e.g. systems where only nss_ldap and pam_ldap are available. For this a new directory server plugin ([https://fedorahosted.org/freeipa/ticket/3567]) will accept the LDAP search request from the legacy systems for the trusted users and groups, resolve the requested objects and send the result back to the legacy client.

Since all trusted users and groups are resolvable on the IPA server via the SSSD IPA provider the idea is that the new plugin will just run getpwnam_r(), getgrnam_r() and related calls. The SSSD disk and memory cache will help to answer those request fast without the need of additional caching inside the directory server.

To offer reliable group lookups to legacy systems it must be possible to lookup all the members of a group from a trusted domain and not only show members which already logged in once on the FreeIPA server, which is the current status on IPA clients with a recent version of SSSD. Additionally legacy systems tend to rely on user and group enumerations. Both requirements force an enumeration and caching of all trusted users and groups on the FreeIPA server.

If the legacy systems used an algorithmic mapping scheme based on the RID of the AD object and an offset to find a POSIX ID for the trusted user or group the ''--base-id'' of the ''ipa trust-add'' command can be used to get the same ID mapping. For legacy systems which read the POSIX IDs directly from AD a new idrange type must be introduced on the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3647]) to indicate that for those trusted users an groups the POSIX ID must be read from AD.

All of the above can basically be solved with the current layout of the FreeIPA server where winbind is doing the lookups against AD and SSSD is using the extdom LDAP plugin to read this data via the directory server. But it was decided to enhance SSSD to do the lookup. Some of the reasons are:
* resources, since SSSD has to run anyway on the FreeIPA server and is capable of the AD user and group lookups, winbind does not have to run anymore
* avoid double caching, to work efficiently winbind has to do some caching on its own and as a result users and groups are cached twice on the FreeIPA server
* configuration, winbind uses a separate configuration file while the IPA provider of SSSD can read e.g. the idranges directly from the FreeIPA server, this minimized to configuration effort and avoids conflicting configuration of different components

=== Overview of the solution ===

First sssd needs to know that it is running on an IPA server and should not look up trusted users and groups with the help of the extdom plugin but do the lookups on its own. For this a new boolean configuration option, e.g. ipa_server_mode, should be introduced ('''SSSD ticket#''') which defaults to ''false'' but is set to ''true'' during ipa-server-install or during updates of the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3652]) if it is not already set.

Since AD by default requires an authenticate LDAP bind to do searches SSSD needs credentials which are accepted by a trusted AD server. Because if the trust relationship this can even be credentials from the FreeIPA domain if Kerberos is user for authentication. So the easiest way is just to use the local keytab which requires no changes on the SSSD side, because the generic LDAP provider already knows how to handle SASL bind with the local keytab.
But currently AD LDAP server does not accept the Kerberos ticket from a FreeIPA host, because the FreeIPA KDC does not attach a PAC to the TGTs of host/ principals ([https://fedorahosted.org/freeipa/ticket/3651], until this is fixed some dummy credentials, e.g. a keytab for a dummy user, can be used).

Now the AD provider code can be used to lookup up the users and group of the trusted AD domain. Only the ID-mapping logic should be refactored so that the same code can be used in the standalone AD provider where the configuration is read form ssd.conf and as part of the IPA provider where the idrange objects read from the IPA server dictates the mapping. Maybe libsss_idmap can be extended to handle idranges for mappings in AD as well, e.g. a specific error code can be used to indicate to the caller that for this domain no algorithmic mapping is available and the value from the corresponding AD attribute should be use ('''SSSD ticket?''').

A task (or a separate process) must be created to handle enumerations efficiently without having to much impact on parallel running requests ('''SSSD ticket#'''). Maybe we can find a scheme which allows to read only a limited (configurable) number of users with their group memberships at a time. This way the cache might not be complete at once but always consistent with respect to group memberships of the caches users. If eventually all users are read, the task will periodically look for new users and update old entries.

=== Implementation details ===
==== Add ipa_server_mode option ====
A new boolean option ipa_server_mode which defaults to false should be added to the IPA provider. ipa_get_subdom_acct_send() should only be called if ipa_server_mode is false. If ipa_server_mode is true ipa_account_info_handler() should return ENOSYS for subdomain requests. A suitable tevent request will be handled in a different ticket.

==== Enhance libsss_idmap ====
1. Allow algorithmic mapping where the first RID is not 0
   Currently it is implicitly assumed that the first POSIX ID of a range is mapped to the RID 0. To support multiple ranges for a single domain a different first RID must handled as well.[[BR]]
   Ticket: #1938
2. Add a range type to handle mappings in AD

==== Add plugin to LDAP provider to find new ranges ====
Currently the range management code is in the generic LDAP provider and can be used by the LDAP and AD provider. New ranges are allocated with the help of a hash value of the domain SID.

If the IPA provider cannot find a range for a given domain it cannot allocate a new range on its own but has to look up the idrange objects on the FreeIPA server and use them accordingly. To allow the LDAP, AD and IPA provider to use as much common code as possible I think a plugin interface, similar to the one used to find the DNS site, to find a missing range would be useful. The default plugin will be used by the LDAP and the AD provider and the IPA provider will implement a plugin to read the data from the server.

==== Integrate AD provider lookup code into IPA subdomain user lookup ====
If the ipa_server_mode is selected IPA subdomain user and group lookups should not be done with the help of the extdom plugin but directly against AD with the help of LDAP of GC lookups. For this the IPA provider must be able to call the related functions from the AD provider. Since by default the POSIX attributes are not replicated to the global catalog and supporting them is a requirement, I think it would be sufficient make sure LDAP lookups are working as expected. Additionally FreeIPA currently supports only one trusted domain global catalog lookups for users and groups from the forest or different forests can be added later.

Since the Kerberos hosts keys from the host keytab should be used as credentials to access AD no changes are expected here.

It should be taken care that not accidentally the the AD SRV plugin is loaded, see next section as well.

==== Enhance IPA SRV plugin to do AD site lookups as well ====
> From the AD point of view trusted domains do not belong to a specific site. But recent version of AD return the next_closest_site for host which do not belong to a site. To make sure that SSSD is communication with an AD server which is network-wise reasonably near it would be useful if the IPA SRV plugin can be enhanced to do CLDAP pings and AD site lookups as well. Additionally the plugin must know when to use IPA style and when AD style lookups.

This is a nice to have feature.

==== Implement or Improve enumeration ====
If enumeration is enable SSSD tries to update all users and groups at startup. As a result the startup time where SSSD is basically blocked and cannot serve requests even for data in the cache can be quite long. A new tevent_req task should be created which can read users and groups from the AD domain in smaller chunks so that other request can always slip in between.

The task should make sure all users and groups are read after a while without reading objects twice in a single run. Maybe it is possible to add a special paged-search tevent request which returns after the first page is read to the caller (instead of doing the paging behind the scenes) which the results and a handle which would allow to continue the the search with the next page? If this is a way to go creating this new request would be another development subtask.

Additionally it has to be considered how to handle large groups. But since we have to read all user as well it might be possible to just read the group memberships of the user and build up the groups in the SSSD cache and let the getgrp*() calls only return entries from the cache and never go to the server directly.

=== How to test ===
If the ipa_server_mode is enable on a FreeIPA server which trusts an AD server, ''getent passwd AD\username'' or ''id AD\username'' should return the expected results for users and groups.

''getent group AD\groupname'' should return results depending the state of enumeration. Immediately after startup with an empty cache e.g. the 'Domain User' group should only have a few members if any. After some time more and more members should be displayed until the enumeration is complete and all users and groups are in the SSSD cache.

=== Author(s) ===
Sumit Bose

[Jakub Hrozek]

On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> Hi,
>
> I have created a design page for one of the next major features of SSSD
> at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> basic idea is that if SSSD is running on a FreeIPA server it should help
> the FreeIPA server to look up users and groups from trusted domains.
>
> For your convenience the content can be found below as well.
>
> Comments are suggestions are welcome.
>
> bye,
> Sumit
>
> == IPA Server Mode ==
> Related tickets:
> [https://fedorahosted.org/sssd/ticket/1821 RFE Allow using UIDs and GIDs from AD in trust case]
> [https://fedorahosted.org/sssd/ticket/1881 RFE Determine how to map SID to UID/GID based on IdM server configuration]
> more to come

Hi,

I like the design page, I think it sums up everything we talked about
earlier.
> === Problem Statement ===
> FreeIPA is planning to make users and groups from trusted domains available to legacy systems, e.g. systems where only nss_ldap and pam_ldap are available. For this a new directory server plugin ([https://fedorahosted.org/freeipa/ticket/3567]) will accept the LDAP search request from the legacy systems for the trusted users and groups, resolve the requested objects and send the result back to the legacy client.
>
> Since all trusted users and groups are resolvable on the IPA server via the SSSD IPA provider the idea is that the new plugin will just run getpwnam_r(), getgrnam_r() and related calls. The SSSD disk and memory cache will help to answer those request fast without the need of additional caching inside the directory server.
>
> To offer reliable group lookups to legacy systems it must be possible to lookup all the members of a group from a trusted domain and not only show members which already logged in once on the FreeIPA server, which is the current status on IPA clients with a recent version of SSSD. Additionally legacy systems tend to rely on user and group enumerations. Both requirements force an enumeration and caching of all trusted users and groups on the FreeIPA server.
>
> If the legacy systems used an algorithmic mapping scheme based on the RID of the AD object and an offset to find a POSIX ID for the trusted user or group the ''--base-id'' of the ''ipa trust-add'' command can be used to get the same ID mapping. For legacy systems which read the POSIX IDs directly from AD a new idrange type must be introduced on the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3647]) to indicate that for those trusted users an groups the POSIX ID must be read from AD.
>
> All of the above can basically be solved with the current layout of the FreeIPA server where winbind is doing the lookups against AD and SSSD is using the extdom LDAP plugin to read this data via the directory server. But it was decided to enhance SSSD to do the lookup. Some of the reasons are:
> resources, since SSSD has to run anyway on the FreeIPA server and is capable of the AD user and group lookups, winbind does not have to run anymore
> avoid double caching, to work efficiently winbind has to do some caching on its own and as a result users and groups are cached twice on the FreeIPA server
> configuration, winbind uses a separate configuration file while the IPA provider of SSSD can read e.g. the idranges directly from the FreeIPA server, this minimized to configuration effort and avoids conflicting configuration of different components
>
> === Overview of the solution ===
>
> First sssd needs to know that it is running on an IPA server and should not look up trusted users and groups with the help of the extdom plugin but do the lookups on its own. For this a new boolean configuration option, e.g. ipa_server_mode, should be introduced ('''SSSD ticket#''') which defaults to ''false'' but is set to ''true'' during ipa-server-install or during updates of the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3652]) if it is not already set.
>
> Since AD by default requires an authenticate LDAP bind to do searches SSSD needs credentials which are accepted by a trusted AD server. Because if the trust relationship this can even be credentials from the FreeIPA domain if Kerberos is user for authentication. So the easiest way is just to use the local keytab which requires no changes on the SSSD side, because the generic LDAP provider already knows how to handle SASL bind with the local keytab.
> But currently AD LDAP server does not accept the Kerberos ticket from a FreeIPA host, because the FreeIPA KDC does not attach a PAC to the TGTs of host/ principals ([https://fedorahosted.org/freeipa/ticket/3651], until this is fixed some dummy credentials, e.g. a keytab for a dummy user, can be used).
>
> Now the AD provider code can be used to lookup up the users and group of the trusted AD domain. Only the ID-mapping logic should be refactored so that the same code can be used in the standalone AD provider where the configuration is read form ssd.conf and as part of the IPA provider where the idrange objects read from the IPA server dictates the mapping. Maybe libsss_idmap can be extended to handle idranges for mappings in AD as well, e.g. a specific error code can be used to indicate to the caller that for this domain no algorithmic mapping is available and the value from the corresponding AD attribute should be use ('''SSSD ticket?''').
>
> A task (or a separate process) must be created to handle enumerations efficiently without having to much impact on parallel running requests ('''SSSD ticket#'''). Maybe we can find a scheme which allows to read only a limited (configurable) number of users with their group memberships at a time. This way the cache might not be complete at once but always consistent with respect to group memberships of the caches users. If eventually all users are read, the task will periodically look for new users and update old entries.
>
> === Implementation details ===
> ==== Add ipa_server_mode option ====
> A new boolean option ipa_server_mode which defaults to false should
> be added to the IPA provider. ipa_get_subdom_acct_send() should only
> be called if ipa_server_mode is false. If ipa_server_mode is true
> ipa_account_info_handler() should return ENOSYS for subdomain requests. A
> suitable tevent request will be handled in a different ticket.

Ack, this one sounds easy. The next ticket that handles the suitable
tevent request is described in integrating the AD provider lookup code,
right?
> ==== Enhance libsss_idmap ====
> Allow algorithmic mapping where the first RID is not 0
> Currently it is implicitly assumed that the first POSIX ID of a range
> is mapped to the RID 0. To support multiple ranges for a single domain a
> different first RID must handled as well.[[BR]]
> Ticket: #1938
> Add a range type to handle mappings in AD

Sorry, it's not quite clear to me what 2. is ? Does it talk about the
plugin interface described also in the next paragraph?
> ==== Add plugin to LDAP provider to find new ranges ====
> Currently the range management code is in the generic LDAP provider and
> can be used by the LDAP and AD provider. New ranges are allocated with
> the help of a hash value of the domain SID.
>
> If the IPA provider cannot find a range for a given domain it cannot
> allocate a new range on its own but has to look up the idrange objects
> on the FreeIPA server and use them accordingly. To allow the LDAP, AD
> and IPA provider to use as much common code as possible I think a plugin
> interface, similar to the one used to find the DNS site, to find a missing
> range would be useful. The default plugin will be used by the LDAP and the
> AD provider and the IPA provider will implement a plugin to read the data
> from the server.

I like this design.
> ==== Integrate AD provider lookup code into IPA subdomain user lookup ====
> If the ipa_server_mode is selected IPA subdomain user and group lookups
> should not be done with the help of the extdom plugin but directly against
> AD with the help of LDAP of GC lookups. For this the IPA provider must
> be able to call the related functions from the AD provider. Since by
> default the POSIX attributes are not replicated to the global catalog
> and supporting them is a requirement, I think it would be sufficient make
> sure LDAP lookups are working as expected. Additionally FreeIPA currently
> supports only one trusted domain global catalog lookups for users and
> groups from the forest or different forests can be added later.

How would the IPA provider know which servers to contact? Does IPA
export this info in some kind of trust config object since the AD site
lookups integration is optional feature?
> Since the Kerberos hosts keys from the host keytab should be used as
> credentials to access AD no changes are expected here.
>
> It should be taken care that not accidentally the the AD SRV plugin is
> loaded, see next section as well.
>
> ==== Enhance IPA SRV plugin to do AD site lookups as well ====
> From the AD point of view trusted domains do not belong to a specific
> site. But recent version of AD return the next_closest_site for host which
> do not belong to a site. To make sure that SSSD is communication with an
> AD server which is network-wise reasonably near it would be useful if the
> IPA SRV plugin can be enhanced to do CLDAP pings and AD site lookups as
> well. Additionally the plugin must know when to use IPA style and when AD
> style lookups.

Maybe we could chain the existing plugins? I think the plugin
architecture would allow mixing different lookup types.
> This is a nice to have feature.

Yes, I agree, I think we could do the design now to see if implementing
this feature would require other changes in the failover code.
> ==== Implement or Improve enumeration ====
> If enumeration is enable SSSD tries to update all users and groups at
> startup. As a result the startup time where SSSD is basically blocked and
> cannot serve requests even for data in the cache can be quite long. A new
> tevent_req task should be created which can read users and groups from the
> AD domain in smaller chunks so that other request can always slip in between.

Maybe we could reuse existing ticket
https://fedorahosted.org/sssd/ticket/1829 for this feature?
> The task should make sure all users and groups are read after a while
> without reading objects twice in a single run. Maybe it is possible to
> add a special paged-search tevent request which returns after the first
> page is read to the caller (instead of doing the paging behind the scenes)
> which the results and a handle which would allow to continue the the search
> with the next page? If this is a way to go creating this new request would
> be another development subtask.

Yes, I was considering about similar design when I was thinking about #1829.

Maybe we could do the enumeration only for the trusted domains so that
it would run on the background only. Then we wouldn't have to worry
about how to handle getpwent/getgrent requests towards the frontend.
> Additionally it has to be considered how to handle large groups. But
> since we have to read all user as well it might be possible to just read
> the group memberships of the user and build up the groups in the SSSD
> cache and let the getgrp*() calls only return entries from the cache and
> never go to the server directly.

As seen in how to test, this would make the SSSD return incomplete group
memberships, isn't that a potential issue for access control?
> === How to test ===
> If the ipa_server_mode is enable on a FreeIPA server which trusts an AD server, ''getent passwd AD\username'' or ''id AD\username'' should return the expected results for users and groups.
>
> ''getent group AD\groupname'' should return results depending the state of enumeration. Immediately after startup with an empty cache e.g. the 'Domain User' group should only have a few members if any. After some time more and more members should be displayed until the enumeration is complete and all users and groups are in the SSSD cache.
>
> === Author(s) ===
> Sumit Bose 
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Sumit Bose]

On Tue, May 28, 2013 at 02:09:54PM +0200, Jakub Hrozek wrote:
> On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > Hi,
> >
> > I have created a design page for one of the next major features of SSSD
> > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > basic idea is that if SSSD is running on a FreeIPA server it should help
> > the FreeIPA server to look up users and groups from trusted domains.
> >
> > For your convenience the content can be found below as well.
> >
> > Comments are suggestions are welcome.
> >
> > bye,
> > Sumit
> >
> > == IPA Server Mode ==
> > Related tickets:
> > [https://fedorahosted.org/sssd/ticket/1821 RFE Allow using UIDs and GIDs from AD in trust case]
> > [https://fedorahosted.org/sssd/ticket/1881 RFE Determine how to map SID to UID/GID based on IdM server configuration]
> > more to come
>
> Hi,
>
> I like the design page, I think it sums up everything we talked about
> earlier.
> > === Problem Statement ===
> > FreeIPA is planning to make users and groups from trusted domains available to legacy systems, e.g. systems where only nss_ldap and pam_ldap are available. For this a new directory server plugin ([https://fedorahosted.org/freeipa/ticket/3567]) will accept the LDAP search request from the legacy systems for the trusted users and groups, resolve the requested objects and send the result back to the legacy client.
> >
> > Since all trusted users and groups are resolvable on the IPA server via the SSSD IPA provider the idea is that the new plugin will just run getpwnam_r(), getgrnam_r() and related calls. The SSSD disk and memory cache will help to answer those request fast without the need of additional caching inside the directory server.
> >
> > To offer reliable group lookups to legacy systems it must be possible to lookup all the members of a group from a trusted domain and not only show members which already logged in once on the FreeIPA server, which is the current status on IPA clients with a recent version of SSSD. Additionally legacy systems tend to rely on user and group enumerations. Both requirements force an enumeration and caching of all trusted users and groups on the FreeIPA server.
> >
> > If the legacy systems used an algorithmic mapping scheme based on the RID of the AD object and an offset to find a POSIX ID for the trusted user or group the ''--base-id'' of the ''ipa trust-add'' command can be used to get the same ID mapping. For legacy systems which read the POSIX IDs directly from AD a new idrange type must be introduced on the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3647]) to indicate that for those trusted users an groups the POSIX ID must be read from AD.
> >
> > All of the above can basically be solved with the current layout of the FreeIPA server where winbind is doing the lookups against AD and SSSD is using the extdom LDAP plugin to read this data via the directory server. But it was decided to enhance SSSD to do the lookup. Some of the reasons are:
> > resources, since SSSD has to run anyway on the FreeIPA server and is capable of the AD user and group lookups, winbind does not have to run anymore
> > avoid double caching, to work efficiently winbind has to do some caching on its own and as a result users and groups are cached twice on the FreeIPA server
> > configuration, winbind uses a separate configuration file while the IPA provider of SSSD can read e.g. the idranges directly from the FreeIPA server, this minimized to configuration effort and avoids conflicting configuration of different components
> >
> > === Overview of the solution ===
> >
> > First sssd needs to know that it is running on an IPA server and should not look up trusted users and groups with the help of the extdom plugin but do the lookups on its own. For this a new boolean configuration option, e.g. ipa_server_mode, should be introduced ('''SSSD ticket#''') which defaults to ''false'' but is set to ''true'' during ipa-server-install or during updates of the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3652]) if it is not already set.
> >
> > Since AD by default requires an authenticate LDAP bind to do searches SSSD needs credentials which are accepted by a trusted AD server. Because if the trust relationship this can even be credentials from the FreeIPA domain if Kerberos is user for authentication. So the easiest way is just to use the local keytab which requires no changes on the SSSD side, because the generic LDAP provider already knows how to handle SASL bind with the local keytab.
> > But currently AD LDAP server does not accept the Kerberos ticket from a FreeIPA host, because the FreeIPA KDC does not attach a PAC to the TGTs of host/ principals ([https://fedorahosted.org/freeipa/ticket/3651], until this is fixed some dummy credentials, e.g. a keytab for a dummy user, can be used).
> >
> > Now the AD provider code can be used to lookup up the users and group of the trusted AD domain. Only the ID-mapping logic should be refactored so that the same code can be used in the standalone AD provider where the configuration is read form ssd.conf and as part of the IPA provider where the idrange objects read from the IPA server dictates the mapping. Maybe libsss_idmap can be extended to handle idranges for mappings in AD as well, e.g. a specific error code can be used to indicate to the caller that for this domain no algorithmic mapping is available and the value from the corresponding AD attribute should be use ('''SSSD ticket?''').
> >
> > A task (or a separate process) must be created to handle enumerations efficiently without having to much impact on parallel running requests ('''SSSD ticket#'''). Maybe we can find a scheme which allows to read only a limited (configurable) number of users with their group memberships at a time. This way the cache might not be complete at once but always consistent with respect to group memberships of the caches users. If eventually all users are read, the task will periodically look for new users and update old entries.
> >
> > === Implementation details ===
> > ==== Add ipa_server_mode option ====
> > A new boolean option ipa_server_mode which defaults to false should
> > be added to the IPA provider. ipa_get_subdom_acct_send() should only
> > be called if ipa_server_mode is false. If ipa_server_mode is true
> > ipa_account_info_handler() should return ENOSYS for subdomain requests. A
> > suitable tevent request will be handled in a different ticket.
>
> Ack, this one sounds easy. The next ticket that handles the suitable
> tevent request is described in integrating the AD provider lookup code,
> right?

yes
> > ==== Enhance libsss_idmap ====
> > Allow algorithmic mapping where the first RID is not 0
> > Currently it is implicitly assumed that the first POSIX ID of a range
> > is mapped to the RID 0. To support multiple ranges for a single domain a
> > different first RID must handled as well.[[BR]]
> > Ticket: #1938
> > Add a range type to handle mappings in AD
>
> Sorry, it's not quite clear to me what 2. is ? Does it talk about the
> plugin interface described also in the next paragraph?

no, my idea is that ranges for IDs from AD can be used in libsss_idmap
as well, but whenever a mapping is requested for this range a specific
error code like IDMAP_ASK_AD_FOR_MAPPING is returned to tell SSSD to do
an AD lookup. This way SSSD does not need to inspect the ranges itself
but all is done inside if libsss_idmap.
> > ==== Add plugin to LDAP provider to find new ranges ====
> > Currently the range management code is in the generic LDAP provider and
> > can be used by the LDAP and AD provider. New ranges are allocated with
> > the help of a hash value of the domain SID.
> >
> > If the IPA provider cannot find a range for a given domain it cannot
> > allocate a new range on its own but has to look up the idrange objects
> > on the FreeIPA server and use them accordingly. To allow the LDAP, AD
> > and IPA provider to use as much common code as possible I think a plugin
> > interface, similar to the one used to find the DNS site, to find a missing
> > range would be useful. The default plugin will be used by the LDAP and the
> > AD provider and the IPA provider will implement a plugin to read the data
> > from the server.
>
> I like this design.
> > ==== Integrate AD provider lookup code into IPA subdomain user lookup ====
> > If the ipa_server_mode is selected IPA subdomain user and group lookups
> > should not be done with the help of the extdom plugin but directly against
> > AD with the help of LDAP of GC lookups. For this the IPA provider must
> > be able to call the related functions from the AD provider. Since by
> > default the POSIX attributes are not replicated to the global catalog
> > and supporting them is a requirement, I think it would be sufficient make
> > sure LDAP lookups are working as expected. Additionally FreeIPA currently
> > supports only one trusted domain global catalog lookups for users and
> > groups from the forest or different forests can be added later.
>
> How would the IPA provider know which servers to contact? Does IPA
> export this info in some kind of trust config object since the AD site
> lookups integration is optional feature?

The IPA trust objects only have the DNS domain name. So we need SRV
lookups to find a DC. I think if we do not get the AD SRV plugin working
in here it is acceptable to do plain SRV lookups, but it would be nice
if we would be able to use the next_closest_site here.
> > Since the Kerberos hosts keys from the host keytab should be used as
> > credentials to access AD no changes are expected here.
> >
> > It should be taken care that not accidentally the the AD SRV plugin is
> > loaded, see next section as well.
> >
> > ==== Enhance IPA SRV plugin to do AD site lookups as well ====
> > From the AD point of view trusted domains do not belong to a specific
> > site. But recent version of AD return the next_closest_site for host which
> > do not belong to a site. To make sure that SSSD is communication with an
> > AD server which is network-wise reasonably near it would be useful if the
> > IPA SRV plugin can be enhanced to do CLDAP pings and AD site lookups as
> > well. Additionally the plugin must know when to use IPA style and when AD
> > style lookups.
>
> Maybe we could chain the existing plugins? I think the plugin
> architecture would allow mixing different lookup types.
> > This is a nice to have feature.
>
> Yes, I agree, I think we could do the design now to see if implementing
> this feature would require other changes in the failover code.
> > ==== Implement or Improve enumeration ====
> > If enumeration is enable SSSD tries to update all users and groups at
> > startup. As a result the startup time where SSSD is basically blocked and
> > cannot serve requests even for data in the cache can be quite long. A new
> > tevent_req task should be created which can read users and groups from the
> > AD domain in smaller chunks so that other request can always slip in between.
>
> Maybe we could reuse existing ticket
> https://fedorahosted.org/sssd/ticket/1829 for this feature?

yes
> > The task should make sure all users and groups are read after a while
> > without reading objects twice in a single run. Maybe it is possible to
> > add a special paged-search tevent request which returns after the first
> > page is read to the caller (instead of doing the paging behind the scenes)
> > which the results and a handle which would allow to continue the the search
> > with the next page? If this is a way to go creating this new request would
> > be another development subtask.
>
> Yes, I was considering about similar design when I was thinking about #1829.
>
> Maybe we could do the enumeration only for the trusted domains so that
> it would run on the background only. Then we wouldn't have to worry

yes, I thought this task will only lookup trusted domain account. If you
want enumeration for the IPA accounts, if can be switched on with the
enumerate config option.
> about how to handle getpwent/getgrent requests towards the frontend.

I'm not sure what you mean here.
> > Additionally it has to be considered how to handle large groups. But
> > since we have to read all user as well it might be possible to just read
> > the group memberships of the user and build up the groups in the SSSD
> > cache and let the getgrp*() calls only return entries from the cache and
> > never go to the server directly.
>
> As seen in how to test, this would make the SSSD return incomplete group
> memberships, isn't that a potential issue for access control?

The idea is to lookup the users with all their memberships. So group
memberships are always complete with respect to the user. But a group
might not already have all it's members, because the missing members are
not read from the server, hence not know to the system.

Additionally if we get a request for a specific AD user which is not
already in the cache then we have to do an individual lookup of the user
and it's group memberships.

bye,
Sumit
> > === How to test ===
> > If the ipa_server_mode is enable on a FreeIPA server which trusts an AD server, ''getent passwd AD\username'' or ''id AD\username'' should return the expected results for users and groups.
> >
> > ''getent group AD\groupname'' should return results depending the state of enumeration. Immediately after startup with an empty cache e.g. the 'Domain User' group should only have a few members if any. After some time more and more members should be displayed until the enumeration is complete and all users and groups are in the SSSD cache.
> >
> > === Author(s) ===
> > Sumit Bose 
> > _______________________________________________
> > sssd-devel mailing list
> > [e-mail]
> > https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
>
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Jakub Hrozek]

On Wed, May 29, 2013 at 12:51:43PM +0200, Sumit Bose wrote:
> > > The task should make sure all users and groups are read after a while
> > > without reading objects twice in a single run. Maybe it is possible to
> > > add a special paged-search tevent request which returns after the first
> > > page is read to the caller (instead of doing the paging behind the scenes)
> > > which the results and a handle which would allow to continue the the search
> > > with the next page? If this is a way to go creating this new request would
> > > be another development subtask.
> >
> > Yes, I was considering about similar design when I was thinking about #1829.
> >
> > Maybe we could do the enumeration only for the trusted domains so that
> > it would run on the background only. Then we wouldn't have to worry
>
> yes, I thought this task will only lookup trusted domain account. If you
> want enumeration for the IPA accounts, if can be switched on with the
> enumerate config option.
> > about how to handle getpwent/getgrent requests towards the frontend.
>
> I'm not sure what you mean here.

There is some special casing in the nss responder that is enabled when a
domain enumerates. I meant that if the enumeration was only on the
background and only for the trusted domains, the nss responder would act
like there was no enumeration at all and "getent passwd" calls could
just return empty like they do when there is no enumeration at all.

[Sumit Bose]

On Wed, May 29, 2013 at 05:17:37PM +0200, Jakub Hrozek wrote:
> On Wed, May 29, 2013 at 12:51:43PM +0200, Sumit Bose wrote:
> > > > The task should make sure all users and groups are read after a while
> > > > without reading objects twice in a single run. Maybe it is possible to
> > > > add a special paged-search tevent request which returns after the first
> > > > page is read to the caller (instead of doing the paging behind the scenes)
> > > > which the results and a handle which would allow to continue the the search
> > > > with the next page? If this is a way to go creating this new request would
> > > > be another development subtask.
> > >
> > > Yes, I was considering about similar design when I was thinking about #1829.
> > >
> > > Maybe we could do the enumeration only for the trusted domains so that
> > > it would run on the background only. Then we wouldn't have to worry
> >
> > yes, I thought this task will only lookup trusted domain account. If you
> > want enumeration for the IPA accounts, if can be switched on with the
> > enumerate config option.
> > > about how to handle getpwent/getgrent requests towards the frontend.
> >
> > I'm not sure what you mean here.
>
> There is some special casing in the nss responder that is enabled when a
> domain enumerates. I meant that if the enumeration was only on the
> background and only for the trusted domains, the nss responder would act
> like there was no enumeration at all and "getent passwd" calls could
> just return empty like they do when there is no enumeration at all.

ah, I see. Yes, the nss responder is not involved in this enumeration.

bye,
Sumit
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Dmitri Pal]

On 05/28/2013 07:20 AM, Sumit Bose wrote:
> Hi,
>
> I have created a design page for one of the next major features of SSSD
> at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> basic idea is that if SSSD is running on a FreeIPA server it should help
> the FreeIPA server to look up users and groups from trusted domains.
>
> For your convenience the content can be found below as well.
>
> Comments are suggestions are welcome.
>
> bye,
> Sumit
>
> == IPA Server Mode ==
> Related tickets:
> [https://fedorahosted.org/sssd/ticket/1821 RFE Allow using UIDs and GIDs from AD in trust case]
> [https://fedorahosted.org/sssd/ticket/1881 RFE Determine how to map SID to UID/GID based on IdM server configuration]
> more to come
>
> === Problem Statement ===
> FreeIPA is planning to make users and groups from trusted domains available to legacy systems, e.g. systems where only nss_ldap and pam_ldap are available. For this a new directory server plugin ([https://fedorahosted.org/freeipa/ticket/3567]) will accept the LDAP search request from the legacy systems for the trusted users and groups, resolve the requested objects and send the result back to the legacy client.
>
> Since all trusted users and groups are resolvable on the IPA server via the SSSD IPA provider the idea is that the new plugin will just run getpwnam_r(), getgrnam_r() and related calls. The SSSD disk and memory cache will help to answer those request fast without the need of additional caching inside the directory server.
>
> To offer reliable group lookups to legacy systems it must be possible to lookup all the members of a group from a trusted domain and not only show members which already logged in once on the FreeIPA server, which is the current status on IPA clients with a recent version of SSSD. Additionally legacy systems tend to rely on user and group enumerations. Both requirements force an enumeration and caching of all trusted users and groups on the FreeIPA server.
>
> If the legacy systems used an algorithmic mapping scheme based on the RID of the AD object and an offset to find a POSIX ID for the trusted user or group the ''--base-id'' of the ''ipa trust-add'' command can be used to get the same ID mapping. For legacy systems which read the POSIX IDs directly from AD a new idrange type must be introduced on the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3647]) to indicate that for those trusted users an groups the POSIX ID must be read from AD.
>
> All of the above can basically be solved with the current layout of the FreeIPA server where winbind is doing the lookups against AD and SSSD is using the extdom LDAP plugin to read this data via the directory server. But it was decided to enhance SSSD to do the lookup. Some of the reasons are:
> resources, since SSSD has to run anyway on the FreeIPA server and is capable of the AD user and group lookups, winbind does not have to run anymore
> avoid double caching, to work efficiently winbind has to do some caching on its own and as a result users and groups are cached twice on the FreeIPA server
> configuration, winbind uses a separate configuration file while the IPA provider of SSSD can read e.g. the idranges directly from the FreeIPA server, this minimized to configuration effort and avoids conflicting configuration of different components
>
> === Overview of the solution ===
>
> First sssd needs to know that it is running on an IPA server and should not look up trusted users and groups with the help of the extdom plugin but do the lookups on its own. For this a new boolean configuration option, e.g. ipa_server_mode, should be introduced ('''SSSD ticket#''') which defaults to ''false'' but is set to ''true'' during ipa-server-install or during updates of the FreeIPA server ([https://fedorahosted.org/freeipa/ticket/3652]) if it is not already set.
>
> Since AD by default requires an authenticate LDAP bind to do searches SSSD needs credentials which are accepted by a trusted AD server. Because if the trust relationship this can even be credentials from the FreeIPA domain if Kerberos is user for authentication. So the easiest way is just to use the local keytab which requires no changes on the SSSD side, because the generic LDAP provider already knows how to handle SASL bind with the local keytab.
> But currently AD LDAP server does not accept the Kerberos ticket from a FreeIPA host, because the FreeIPA KDC does not attach a PAC to the TGTs of host/ principals ([https://fedorahosted.org/freeipa/ticket/3651], until this is fixed some dummy credentials, e.g. a keytab for a dummy user, can be used).
>
> Now the AD provider code can be used to lookup up the users and group of the trusted AD domain. Only the ID-mapping logic should be refactored so that the same code can be used in the standalone AD provider where the configuration is read form ssd.conf and as part of the IPA provider where the idrange objects read from the IPA server dictates the mapping. Maybe libsss_idmap can be extended to handle idranges for mappings in AD as well, e.g. a specific error code can be used to indicate to the caller that for this domain no algorithmic mapping is available and the value from the corresponding AD attribute should be use ('''SSSD ticket?''').
>
> A task (or a separate process) must be created to handle enumerations efficiently without having to much impact on parallel running requests ('''SSSD ticket#'''). Maybe we can find a scheme which allows to read only a limited (configurable) number of users with their group memberships at a time. This way the cache might not be complete at once but always consistent with respect to group memberships of the caches users. If eventually all users are read, the task will periodically look for new users and update old entries.
>
> === Implementation details ===
> ==== Add ipa_server_mode option ====
> A new boolean option ipa_server_mode which defaults to false should be added to the IPA provider. ipa_get_subdom_acct_send() should only be called if ipa_server_mode is false. If ipa_server_mode is true ipa_account_info_handler() should return ENOSYS for subdomain requests. A suitable tevent request will be handled in a different ticket.
>
> ==== Enhance libsss_idmap ====
> Allow algorithmic mapping where the first RID is not 0
> Currently it is implicitly assumed that the first POSIX ID of a range is mapped to the RID 0. To support multiple ranges for a single domain a different first RID must handled as well.[[BR]]
> Ticket: #1938
> Add a range type to handle mappings in AD
>
> ==== Add plugin to LDAP provider to find new ranges ====
> Currently the range management code is in the generic LDAP provider and can be used by the LDAP and AD provider. New ranges are allocated with the help of a hash value of the domain SID.
>
> If the IPA provider cannot find a range for a given domain it cannot allocate a new range on its own but has to look up the idrange objects on the FreeIPA server and use them accordingly. To allow the LDAP, AD and IPA provider to use as much common code as possible I think a plugin interface, similar to the one used to find the DNS site, to find a missing range would be useful. The default plugin will be used by the LDAP and the AD provider and the IPA provider will implement a plugin to read the data from the server.
>
> ==== Integrate AD provider lookup code into IPA subdomain user lookup ====
> If the ipa_server_mode is selected IPA subdomain user and group lookups should not be done with the help of the extdom plugin but directly against AD with the help of LDAP of GC lookups. For this the IPA provider must be able to call the related functions from the AD provider. Since by default the POSIX attributes are not replicated to the global catalog and supporting them is a requirement, I think it would be sufficient make sure LDAP lookups are working as expected. Additionally FreeIPA currently supports only one trusted domain global catalog lookups for users and groups from the forest or different forests can be added later.
>
> Since the Kerberos hosts keys from the host keytab should be used as credentials to access AD no changes are expected here.
>
> It should be taken care that not accidentally the the AD SRV plugin is loaded, see next section as well.
>
> ==== Enhance IPA SRV plugin to do AD site lookups as well ====
> From the AD point of view trusted domains do not belong to a specific site. But recent version of AD return the next_closest_site for host which do not belong to a site. To make sure that SSSD is communication with an AD server which is network-wise reasonably near it would be useful if the IPA SRV plugin can be enhanced to do CLDAP pings and AD site lookups as well. Additionally the plugin must know when to use IPA style and when AD style lookups.
>
> This is a nice to have feature.
>
> ==== Implement or Improve enumeration ====
> If enumeration is enable SSSD tries to update all users and groups at startup. As a result the startup time where SSSD is basically blocked and cannot serve requests even for data in the cache can be quite long. A new tevent_req task should be created which can read users and groups from the AD domain in smaller chunks so that other request can always slip in between.
>
> The task should make sure all users and groups are read after a while without reading objects twice in a single run. Maybe it is possible to add a special paged-search tevent request which returns after the first page is read to the caller (instead of doing the paging behind the scenes) which the results and a handle which would allow to continue the the search with the next page? If this is a way to go creating this new request would be another development subtask.
>
> Additionally it has to be considered how to handle large groups. But since we have to read all user as well it might be possible to just read the group memberships of the user and build up the groups in the SSSD cache and let the getgrp*() calls only return entries from the cache and never go to the server directly.
>
> === How to test ===
> If the ipa_server_mode is enable on a FreeIPA server which trusts an AD server, ''getent passwd AD\username'' or ''id AD\username'' should return the expected results for users and groups.
>
> ''getent group AD\groupname'' should return results depending the state of enumeration. Immediately after startup with an empty cache e.g. the 'Domain User' group should only have a few members if any. After some time more and more members should be displayed until the enumeration is complete and all users and groups are in the SSSD cache.
>
> === Author(s) ===
> Sumit Bose 
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

The ipa_server_mode should be IMO a bit more sophisticated.
It should not do anything until it sees that IPA participates in any
trust relationship with AD. AFAIU there are ways on the SSSD side to
detect that there is a trust added on IPA side.
So the logic should be something like:

if ipa_server_mode = false then we are on a client, no change
else
  if we already detected that there is at least one trusted domain
      continue with the logic described above
  else
      when there is a request for the user check if any trusts are
available
      if they are set the flag that the trusts are detected and execute
the logic
      else behave as if ipa_server_mode is false

[Sumit Bose]

On Tue, May 28, 2013 at 03:11:20PM -0400, Dmitri Pal wrote:
> On 05/28/2013 07:20 AM, Sumit Bose wrote:
> > Hi,
> >
> > I have created a design page for one of the next major features of SSSD
> > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > basic idea is that if SSSD is running on a FreeIPA server it should help
> > the FreeIPA server to look up users and groups from trusted domains.
> >
> > For your convenience the content can be found below as well.
> >
> > Comments are suggestions are welcome.
> >
> > bye,
> > Sumit

...
> The ipa_server_mode should be IMO a bit more sophisticated.
> It should not do anything until it sees that IPA participates in any
> trust relationship with AD. AFAIU there are ways on the SSSD side to
> detect that there is a trust added on IPA side.
> So the logic should be something like:
>
> if ipa_server_mode = false then we are on a client, no change
> else
>   if we already detected that there is at least one trusted domain
>       continue with the logic described above
>   else
>       when there is a request for the user check if any trusts are
> available
>       if they are set the flag that the trusts are detected and execute
> the logic
>       else behave as if ipa_server_mode is false

This is basically already the case because the nss provider will only forward
requests for known trusted domains.

Nevertheless I think you raised a good point for the enumeration task. I
just thought that after startup it regularly looks if there are trusted
domains. But it would be better if it stops if it cannot find any and
will be restarted as soon as trusted domains are discovered.

bye,
Sumit
> -- 
> Thank you,
> Dmitri Pal
>
> Sr. Engineering Manager for IdM portfolio
> Red Hat Inc.
>
> Looking to carve out IT costs?
> www.redhat.com/carveoutcosts/
>
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Dmitri Pal]

On 05/29/2013 07:09 AM, Sumit Bose wrote:
> On Tue, May 28, 2013 at 03:11:20PM -0400, Dmitri Pal wrote:
> > On 05/28/2013 07:20 AM, Sumit Bose wrote:
> > > Hi,
> > >
> > > I have created a design page for one of the next major features of SSSD
> > > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > > basic idea is that if SSSD is running on a FreeIPA server it should help
> > > the FreeIPA server to look up users and groups from trusted domains.
> > >
> > > For your convenience the content can be found below as well.
> > >
> > > Comments are suggestions are welcome.
> > >
> > > bye,
> > > Sumit
>
> ...
> > The ipa_server_mode should be IMO a bit more sophisticated.
> > It should not do anything until it sees that IPA participates in any
> > trust relationship with AD. AFAIU there are ways on the SSSD side to
> > detect that there is a trust added on IPA side.
> > So the logic should be something like:
> >
> > if ipa_server_mode = false then we are on a client, no change
> > else
> >   if we already detected that there is at least one trusted domain
> >       continue with the logic described above
> >   else
> >       when there is a request for the user check if any trusts are
> > available
> >       if they are set the flag that the trusts are detected and execute
> > the logic
> >       else behave as if ipa_server_mode is false
>
> This is basically already the case because the nss provider will only forward
> requests for known trusted domains.
>
> Nevertheless I think you raised a good point for the enumeration task. I
> just thought that after startup it regularly looks if there are trusted
> domains. But it would be better if it stops if it cannot find any and
> will be restarted as soon as trusted domains are discovered.

Yes. Thanks.
> bye,
> Sumit
> > -- 
> > Thank you,
> > Dmitri Pal
> >
> > Sr. Engineering Manager for IdM portfolio
> > Red Hat Inc.
> >
> > Looking to carve out IT costs?
> > www.redhat.com/carveoutcosts/
> >
> > sssd-devel mailing list
> > [e-mail]
> > https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
>
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel

[Sumit Bose]

On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> Hi,
>
> I have created a design page for one of the next major features of SSSD
> at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> basic idea is that if SSSD is running on a FreeIPA server it should help
> the FreeIPA server to look up users and groups from trusted domains.
>
> For your convenience the content can be found below as well.
>
> Comments are suggestions are welcome.
>
> bye,
> Sumit

Thanks to Jakub and Dmitri for their suggestions. I have updated the
page accordingly. Comments are suggestions are still welcome :-)

bye,
Sumit

[Jakub Hrozek]

On Thu, May 30, 2013 at 02:07:00PM +0200, Sumit Bose wrote:
> On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > Hi,
> >
> > I have created a design page for one of the next major features of SSSD
> > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > basic idea is that if SSSD is running on a FreeIPA server it should help
> > the FreeIPA server to look up users and groups from trusted domains.
> >
> > For your convenience the content can be found below as well.
> >
> > Comments are suggestions are welcome.
> >
> > bye,
> > Sumit
>
> Thanks to Jakub and Dmitri for their suggestions. I have updated the
> page accordingly. Comments are suggestions are still welcome :-)
>
> bye,
> Sumit

We had a discussion in the morning with Alexander on #sssd. He suggested
that the SSSD might expose the list of trusted AD domains maybe via
libnss_sss_idmap. I think this would work and this library would be the
good place to add the functionality.

Also I will open a new ticket to store additional information about
trusted domains that IPA needs.

[Alexander Bokovoy]

On Thu, 30 May 2013, Jakub Hrozek wrote:
> On Thu, May 30, 2013 at 02:07:00PM +0200, Sumit Bose wrote:
> > On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > > Hi,
> > >
> > > I have created a design page for one of the next major features of SSSD
> > > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > > basic idea is that if SSSD is running on a FreeIPA server it should help
> > > the FreeIPA server to look up users and groups from trusted domains.
> > >
> > > For your convenience the content can be found below as well.
> > >
> > > Comments are suggestions are welcome.
> > >
> > > bye,
> > > Sumit
> >
> > Thanks to Jakub and Dmitri for their suggestions. I have updated the
> > page accordingly. Comments are suggestions are still welcome :-)
> >
> > bye,
> > Sumit
>
> We had a discussion in the morning with Alexander on #sssd. He suggested
> that the SSSD might expose the list of trusted AD domains maybe via
> libnss_sss_idmap. I think this would work and this library would be the
> good place to add the functionality.
>
> Also I will open a new ticket to store additional information about
> trusted domains that IPA needs.

The reason I've asked about that API extension is to avoid duplicating
code paths in IPA and SSSD when running on the IPA server to support
transitive trusts in FreeIPA 3.4.

When IPA establishes trust to AD domain, we will need to run a query to
fetch trusted domain objects for the AD forest. This information is
available over secure channel in AD which has unfortunate serialization
requirement. Schannel is controlled by winbindd, we cannot open a
separate schannel as that would make first one obsolete.

So this means IPA would need to resort to fetch TDOs from Global
Catalog, essentially re-creating work that SSSD is putting into 1.11.

[Jakub Hrozek]

On Thu, May 30, 2013 at 05:05:52PM +0300, Alexander Bokovoy wrote:
> On Thu, 30 May 2013, Jakub Hrozek wrote:
> > On Thu, May 30, 2013 at 02:07:00PM +0200, Sumit Bose wrote:
> > > On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > > > Hi,
> > > >
> > > > I have created a design page for one of the next major features of SSSD
> > > > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > > > basic idea is that if SSSD is running on a FreeIPA server it should help
> > > > the FreeIPA server to look up users and groups from trusted domains.
> > > >
> > > > For your convenience the content can be found below as well.
> > > >
> > > > Comments are suggestions are welcome.
> > > >
> > > > bye,
> > > > Sumit
> > >
> > > Thanks to Jakub and Dmitri for their suggestions. I have updated the
> > > page accordingly. Comments are suggestions are still welcome :-)
> > >
> > > bye,
> > > Sumit
> >
> > We had a discussion in the morning with Alexander on #sssd. He suggested
> > that the SSSD might expose the list of trusted AD domains maybe via
> > libnss_sss_idmap. I think this would work and this library would be the
> > good place to add the functionality.
> >
> > Also I will open a new ticket to store additional information about
> > trusted domains that IPA needs.
>
> The reason I've asked about that API extension is to avoid duplicating
> code paths in IPA and SSSD when running on the IPA server to support
> transitive trusts in FreeIPA 3.4.
>
> When IPA establishes trust to AD domain, we will need to run a query to
> fetch trusted domain objects for the AD forest. This information is
> available over secure channel in AD which has unfortunate serialization
> requirement. Schannel is controlled by winbindd, we cannot open a
> separate schannel as that would make first one obsolete.
>
> So this means IPA would need to resort to fetch TDOs from Global
> Catalog, essentially re-creating work that SSSD is putting into 1.11.

OK, thank you for the clarification, I filed two tickets to track these:
https://fedorahosted.org/sssd/ticket/1957
https://fedorahosted.org/sssd/ticket/1958

But since these are not needed to reach the goal of 1.11, I suspect the
enhancements will be implemented in 1.12.

[Sumit Bose]

On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> Hi,
>
> I have created a design page for one of the next major features of SSSD
> at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> basic idea is that if SSSD is running on a FreeIPA server it should help
> the FreeIPA server to look up users and groups from trusted domains.
>
> For your convenience the content can be found below as well.
>
> Comments are suggestions are welcome.
>
> bye,
> Sumit

based on a discussion on the freeipa-devel mailing list I've add an
additional section to 'Implementation details':

Remove assumption that subdomain users always have a primary
user-private-group (UPG)

Currently the PAC responder assumes that subdomain users always have a
UPG as primary group. This will be only true for domains with
algorithmic mappings because here the POSIX IDs are managed by the
FreeIPA server and we are free to choose. But if the POSIX IDs are
manged externally we have to use what we get from external sources. E.g.
in the case where the POSIX IDs are managed by AD UIDs and GIDs are
separate name spaces and assuming the UPGs can be used would most
certainly lead to GID conflicts. The PAC responder has to respect the
idrange type or the mpg flag of the sss_domain_info struct and act
accordingly.

Additional the code paths where new subdomains are created must be
reviewed and whereever the mpg flag is set code must be added so that it
is set according to the range type.

Although I think that the code path where an IPA client (i.e.
ipa_server-mode = false) looks up a trusted domain user adds the user to
the cache with the data it receives from the extdom plugin, it should be
verified that UPGs are not implicitly assumed here as well.

bye,
Sumit

[Simo Sorce]

On Wed, 2013-06-05 at 18:31 +0200, Sumit Bose wrote:
> On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > Hi,
> >
> > I have created a design page for one of the next major features of SSSD
> > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > basic idea is that if SSSD is running on a FreeIPA server it should help
> > the FreeIPA server to look up users and groups from trusted domains.
> >
> > For your convenience the content can be found below as well.
> >
> > Comments are suggestions are welcome.
> >
> > bye,
> > Sumit
>
> based on a discussion on the freeipa-devel mailing list I've add an
> additional section to 'Implementation details':
>
> Remove assumption that subdomain users always have a primary
> user-private-group (UPG)
>
> Currently the PAC responder assumes that subdomain users always have a
> UPG as primary group. This will be only true for domains with
> algorithmic mappings because here the POSIX IDs are managed by the
> FreeIPA server and we are free to choose. But if the POSIX IDs are
> manged externally we have to use what we get from external sources. E.g.
> in the case where the POSIX IDs are managed by AD UIDs and GIDs are
> separate name spaces and assuming the UPGs can be used would most
> certainly lead to GID conflicts. The PAC responder has to respect the
> idrange type or the mpg flag of the sss_domain_info struct and act
> accordingly.
>
> Additional the code paths where new subdomains are created must be
> reviewed and whereever the mpg flag is set code must be added so that it
> is set according to the range type.
>
> Although I think that the code path where an IPA client (i.e.
> ipa_server-mode = false) looks up a trusted domain user adds the user to
> the cache with the data it receives from the extdom plugin, it should be
> verified that UPGs are not implicitly assumed here as well.

It seem to me that what we are ending up with is that we need to come to
a final situation where mpg is only ever used at store time, but at
retrieve time it shouldn't

This will allow the cache to be authoritative and hold whatever matters.
We can do this w/o loosing some cache optimization by adding an
attribute to user objects if mpg was assumed at store time, and not
adding the flag if not. (the flag can be as simple as 'objectclass:
group' if nothing else breaks this way).

Simo.

[Sumit Bose]

On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> Hi,
>
> I have created a design page for one of the next major features of SSSD
> at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> basic idea is that if SSSD is running on a FreeIPA server it should help
> the FreeIPA server to look up users and groups from trusted domains.
>
> For your convenience the content can be found below as well.
>
> Comments are suggestions are welcome.
>
> bye,
> Sumit

I added bullet points 3 and 4 to 'Enhance libsss_idmap' and extended
number 2. If there is no disagreement I will open/extended the related
tickets in the evening.

bye,
Sumit

Enhance libsss_idmap

1. Allow algorithmic mapping where the first RID is not 0
Currently it is implicitly assumed that the first POSIX ID of a range is mapped to the RID 0. To support multiple ranges for a single domain a different first RID must handled as well.
Ticket: #1938

2. Add a range type to handle mappings in AD
The idea is that ranges for IDs from AD can be used in libsss_idmap as well, but whenever a mapping is requested for this range a specific error code like IDMAP_ASK_AD_FOR_MAPPING is returned to tell SSSD to do an AD lookup. This way SSSD does not need to inspect the ranges itself but all is done inside if libsss_idmap. Additionally a new call is needed to check whether the returned externally managed ID belongs to a configured range, if not the ID cannot be mapped in the given configuration and the related object should be ignored.
Ticket: #1960

3- Add an optional unique range identifier
To be able to detect configuration changes in idranges managed by FreeIPA an identifier should be stored on the client together with the other idrange related data. For simplicity the DN of the related LDAP object on the FreeIPA server can be used here.

4. Allow updates and removal of ranges
To support configuration changes at runtime, it must be possible to update and remove ranges. As a first step I would recommend that the changes will only affect new requests and not the cached data, because in general changes to centrally manages ranges should be done with care to avoid conflicts. In a later release we can decided if we just want to invalidate all cached entries of the domain which idrange was modified or if a smarter check is needed to invalidate only objects which are affected by the change.

[Jakub Hrozek]

On Mon, Jun 10, 2013 at 11:45:08AM +0200, Sumit Bose wrote:
> On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > Hi,
> >
> > I have created a design page for one of the next major features of SSSD
> > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > basic idea is that if SSSD is running on a FreeIPA server it should help
> > the FreeIPA server to look up users and groups from trusted domains.
> >
> > For your convenience the content can be found below as well.
> >
> > Comments are suggestions are welcome.
> >
> > bye,
> > Sumit
>
> I added bullet points 3 and 4 to 'Enhance libsss_idmap' and extended
> number 2. If there is no disagreement I will open/extended the related
> tickets in the evening.
>
> bye,
> Sumit

Sorry for late response.
> Enhance libsss_idmap
> Allow algorithmic mapping where the first RID is not 0
>
> Currently it is implicitly assumed that the first POSIX ID of a range is mapped to the RID 0. To support multiple ranges for a single domain a different first RID must handled as well.
> Ticket: #1938
> Add a range type to handle mappings in AD
>
> The idea is that ranges for IDs from AD can be used in libsss_idmap as well, but whenever a mapping is requested for this range a specific error code like IDMAP_ASK_AD_FOR_MAPPING is returned to tell SSSD to do an AD lookup. This way SSSD does not need to inspect the ranges itself but all is done inside if libsss_idmap. Additionally a new call is needed to check whether the returned externally managed ID belongs to a configured range, if not the ID cannot be mapped in the given configuration and the related object should be ignored.
> Ticket: #1960

Ack
> 3- Add an optional unique range identifier
> To be able to detect configuration changes in idranges managed by FreeIPA an identifier should be stored on the client together with the other idrange related data. For simplicity the DN of the related LDAP object on the FreeIPA server can be used here.

Ack
> Allow updates and removal of ranges
>
> To support configuration changes at runtime, it must be possible to update and remove ranges. As a first step I would recommend that the changes will only affect new requests and not the cached data, because in general changes to centrally manages ranges should be done with care to avoid conflicts. In a later release we can decided if we just want to invalidate all cached entries of the domain which idrange was modified or if a smarter check is needed to invalidate only objects which are affected by the change.

Do we need to support runtime changes even now? It sounds like a future
enhancement to me..

Also, how exactly (and where) would the configuration changes be
performed? Using some IPA config tool?

[Sumit Bose]

On Wed, Jun 12, 2013 at 06:19:38PM +0200, Jakub Hrozek wrote:
> On Mon, Jun 10, 2013 at 11:45:08AM +0200, Sumit Bose wrote:
> > On Tue, May 28, 2013 at 01:20:15PM +0200, Sumit Bose wrote:
> > > Hi,
> > >
> > > I have created a design page for one of the next major features of SSSD
> > > at https://fedorahosted.org/sssd/wiki/DesignDocs/IPAServerMode . The
> > > basic idea is that if SSSD is running on a FreeIPA server it should help
> > > the FreeIPA server to look up users and groups from trusted domains.
> > >
> > > For your convenience the content can be found below as well.
> > >
> > > Comments are suggestions are welcome.
> > >
> > > bye,
> > > Sumit
> >
> > I added bullet points 3 and 4 to 'Enhance libsss_idmap' and extended
> > number 2. If there is no disagreement I will open/extended the related
> > tickets in the evening.
> >
> > bye,
> > Sumit
>
> Sorry for late response.

np, thank you for the review.
> > Enhance libsss_idmap
> > Allow algorithmic mapping where the first RID is not 0
> >
> > Currently it is implicitly assumed that the first POSIX ID of a range is mapped to the RID 0. To support multiple ranges for a single domain a different first RID must handled as well.
> > Ticket: #1938
> > Add a range type to handle mappings in AD
> >
> > The idea is that ranges for IDs from AD can be used in libsss_idmap as well, but whenever a mapping is requested for this range a specific error code like IDMAP_ASK_AD_FOR_MAPPING is returned to tell SSSD to do an AD lookup. This way SSSD does not need to inspect the ranges itself but all is done inside if libsss_idmap. Additionally a new call is needed to check whether the returned externally managed ID belongs to a configured range, if not the ID cannot be mapped in the given configuration and the related object should be ignored.
> > Ticket: #1960
>
> Ack

updated #1960
> > 3- Add an optional unique range identifier
> > To be able to detect configuration changes in idranges managed by FreeIPA an identifier should be stored on the client together with the other idrange related data. For simplicity the DN of the related LDAP object on the FreeIPA server can be used here.
>
> Ack

created #1979 and assigned it to me, because I already have a patch
> > Allow updates and removal of ranges
> >
> > To support configuration changes at runtime, it must be possible to update and remove ranges. As a first step I would recommend that the changes will only affect new requests and not the cached data, because in general changes to centrally manages ranges should be done with care to avoid conflicts. In a later release we can decided if we just want to invalidate all cached entries of the domain which idrange was modified or if a smarter check is needed to invalidate only objects which are affected by the change.
>
> Do we need to support runtime changes even now? It sounds like a future
> enhancement to me..

yes, the most common use case is to read and include new ranges which we
already cover. Runtime changes should be rare and imo should be only
done with great care.

I created #1981 and put it in 1.11 beta, but marked it as minor, so that
it can be pushed out to a later version.
> Also, how exactly (and where) would the configuration changes be
> performed? Using some IPA config tool?

yes

bye,
Sumit
> sssd-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel