August 2009 - sssd-devel - Fedora mailing-lists

[PATCHES] python bindings for managing users in local domains
by Jakub Hrozek 21 Sep '09

21 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Attached are three patches. [PATCH 1/3] Refactor tools code Refactores tools code while still retaining the original structure. Move parameter parsing in tools before attempting to do anything that might fail - so that we have debug_level set correctly for potential error messages. That allows printing the --help and --usage messages without being root. [PATCH 2/3] Decouple synchronous sysdb interface from tools Instead of working directly with async code in tools, create synchronous wrappers that could be used by tools and python bindings. [PATCH 3/3] Provide python bindings for sysdb Implement a set of python bindings for the sysdb with feature set similar to what is available in the tools. The primary consumers would be applications like system-config-users. These three patches resolve tickets 102, 87 and 95. Jakub -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iEYEARECAAYFAkqNx4UACgkQHsardTLnvCXWSQCgyVRBOCQOPr1Wltg6Kv9/7Bxk C9AAmwcvhQ2VxUeG4uMklgJT19cTzw6L =l7DI -----END PGP SIGNATURE-----

4 23

[PATCH] add change password target to krb5 backend
by Sumit Bose 11 Sep '09

11 Sep '09

Hi, this patch adds the change password support for the kerberos backend. I wonder if we want to support password reset by root via kerberos, i.e. allow root to change a user's password if a krb5 ticket with the needed privileges is available? bye, Sumit

2 3

[PATCH] fix DBusWatch handling
by Simo Sorce 10 Sep '09

10 Sep '09

After some playing around with Stephen patch and adding some debugging I finally understood how DBUS likes to deal with watches. We erroneously assumed that DBUS would add only one watch per FD. It doesn't, instead it always adds 2 watches. One to catch read events and one to catch write events. It then proceed to always keep one of the 2 disabled when the other is enabled, and it always toggles both when it needs to reverse access to the file descriptor. The following patch keeps track of sbus-watches/fd pairs and allocate both DBusWatches to the same structure. So far testing seem to confirm this approach works as expected. Simo. -- Simo Sorce * Red Hat, Inc * New York

3 3

[PATCH] Add support for the EntryCacheNoWaitRefreshTimeout
by Stephen Gallagher 09 Sep '09

09 Sep '09

This timeout specifies the lifetime of a cache entry before it is updated out-of-band. When this timeout is hit, the request will still complete from cache, but the SSSD will also go and update the cached entry in the background to extend the life of the cache entry and reduce the wait time of a future request. Support for the EnumCacheNoWaitRefreshTimeout is still forthcoming, but I wanted to get a formal review on this portion. -- Stephen Gallagher RHCE 804006346421761 Looking to carve out IT costs? www.redhat.com/carveoutcosts/

2 9

[PATCH] ELAPI sinks and providers
by Dmitri Pal 08 Sep '09

08 Sep '09

Please see the patch comment. -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/

3 10

New Try to build SSSD on Karmic both as of 2009.08.04
by Miguel P.C. 08 Sep '09

08 Sep '09

Hi Stephen, hello everyone, I installed the two libraries you said: * docbook-xsl * libxml2-utils And also the ones indicated before: * libtalloc-dev * tdb-dev * libtevent-dev * libldb-dev * cvs * libpopt-dev * libpam-dev * libldap2-dev * libpcre3-dev * krb5-config * libkrb5-dev * libc-ares-dev * libdbus-1-dev * libnss3-dev Hit configure once more: ./configure --prefix=/opt/sssd | tee ../configure.12.log ... and then make: make | tee ../make.1.log I still get the same problem. Seems to be an ldap build dep that doesnt work fine. Do we need 389DS libs to build it? I attach logs. M*

5 14

contribution policy work
by Karsten Wade 31 Aug '09

31 Aug '09

In the process of working on the FreeIPA contribution policy, I am breaking out a separate discussion around SSSD. The goal is to set a contribution policy that the developers of SSSD are happy with, with whatever caveats we have gathered from our lawyer friends. Stephen has been looking in to the question of how to license SSSD contributions, right? I wanted to get us thinking about how to apply those eventual results against a contribution policy. Here are the two drafts I have written for FreeIPA: http://freeipa.org/page/User:Quaid/Contribution_policy_(draft)#License-spec… http://freeipa.org/page/User:Quaid/Contribution_policy_(draft)#License-agno… These are fairly close to being generic enough for use by another project. Both have an "acceptable" stamp from Richard Fontana, depending on what the developers want. The thread with more details about why that policy is written that way is here: https://www.redhat.com/archives/freeipa-devel/2009-August/msg00024.html One thing I think is interesting here is, the lawyer is asking the developers who know the current community, "What is right for you?" I'm not sure if that is normal, but as a community organizer type, I think it is approaching the question from the right direction. :) - Karsten -- Karsten 'quaid' Wade, Community Gardener http://quaid.fedorapeople.org AD0E0C41

1 0

[PATCH] Turn enumeration parameter to boolean
by Simo Sorce 31 Aug '09

31 Aug '09

Since we enumerate in background and use the member/memberof scheme you can't do group enumerations without also doing a user enumeration first. So turn the enumeration option into a boolean with warnings if the old syntax is used. Simo. -- Simo Sorce * Red Hat, Inc * New York

3 4

INI and ELAPI tickets
by Dmitri Pal 28 Aug '09

28 Aug '09

Hi, I have opened several tickets regarding ELAPI and INI. They are tickets 140-143. Please take a look. Note the priorities. Comments and suggestions are welcome. -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/

1 0

[PATCH] check if gid attribute is empty
by Sumit Bose 28 Aug '09

28 Aug '09

Hi, this patch just makes sure that the LDAP backend does not die if the group object returned by the LDAP server does not contain a gid. In a previous patch I have sent the same fix for uid/gid in a returned user object, but have forgotten the group object. bye, Sumit

2 1

[PATCH] fix internal order of ldap user mapping options
by Sumit Bose 28 Aug '09

28 Aug '09

Hi, a previous patch added a new attribute and also changed the internal order of the existing attributes. This patch changes the numbering of the attributes to match the new order. bye, Sumit

2 1

[PATCH] send SSSD_REALM and SSSD_KDCIP environment to the client
by Sumit Bose 28 Aug '09

28 Aug '09

Hi, the environment variable which are send back by this patch are currently needed in the user session of the client. When the DNS helper is available and used by the kerberos locator plugin they can be removed. bye, Sumit

2 2

[PATCH] Speed up NSS enumeration code
by Simo Sorce 28 Aug '09

28 Aug '09

This patch should make the enumeration code ~ O(log n) instead of O(n) On my system it brought enumeration down from 12s to 4s with the same data set. Simo. -- Simo Sorce * Red Hat, Inc * New York

2 2

Re: [SSSD] [Freeipa-devel] [PATCH] add configure check for errno_t
by Sumit Bose 28 Aug '09

28 Aug '09

On Mon, Jul 27, 2009 at 09:53:23AM -0400, Stephen Gallagher wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 07/24/2009 05:05 PM, Sumit Bose wrote: > > Hi, > > > > we regular use functions returning errno error code like ENOMEN or > > EINVAL. The return type of these functions is usually int. To underline > > that the return code is only an error code and should not be used > > otherwise a special type can be used. There is the ISO/ICE technical > > report 24731-1 which recommends they type errno_t for this purpose. I > > think it makes sense to stick with this recommendation (and I hope > > people don't mind that the same TR also recommends the not so smart *_s > > replacements for most of the standard libc functions). > > > > The attached patch add a configure check for errno_t and defines it if > > not present in the system. > > > > bye, > > Sumit > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Freeipa-devel mailing list > > Freeipa-devel(a)redhat.com > > https://www.redhat.com/mailman/listinfo/freeipa-devel > > Nack. > > Please include a patch that modifies the configure scripts of the > sss_client and common directories. > I have include the check in the sss_client directory, but I'm reluctant to add it to common, too. IMO the maintainers of the packages should decide if errno_t makes sense for them or not. A second argument would be that in most cases there is no private header file where the typedef can be added. bye, Sumit

3 4

Some ELAPI design questions
by Dmitri Pal 27 Aug '09

27 Aug '09

Hi, I have some thoughts that I wanted to run by you. May be I am getting too deep and trying to overengineer things. And I am definitely a bit confused about what should I use in the specific case. When writing into the file via ELAPI I need to understand how to deal with: * async processing * buffering * streams and file descriptors The async processing means that we need to be able to prepare data to write and then indicate that we want it to be written but do not block. The actual writing happens from the main loop when the file dgot green light and we know that it would not block. Same with read. The async processing is all based on file descriptors as far as I have seen. When we talk about buffering there are different levels of buffering. There is kernel buffering and stdio stream buffering. Streams are more convenient to use plus I wanted to provide a sink that writes to stderr, but may be I should reconsider. So what is the problem? It seems that stdio and async processing should not be used together. Is this correct? Even if I get fileno() from the stream I write to and pass it to the library like tevent (or similar) the async processing will be screwed by streams buffering data. Is this correct understanding? Can I use stdio or should I use low level : open-write-fsync-close functions? There are "_unlocked" equivalents of some of the stdio functions but it is not clear if they will be available on other platforms. If I do not use stdio I will not take advantage of the stdio buffering. If I understand correctly in this case the fsync forces kernel to flush the page. Is this Ok? If I use stdio then I can potentially have one implementation for writing to file and to stderr. But here is another issue with using stdio and stderr in particular. The program that uses ELAPI can be a daemon which closes file decriptors 0-2. Using close(). In this case trying to use stderr would not do any good. So before using stderr I need to check fd 2 is still open and if it is not do something about it. The expectation and the goal is to effectively recover the stderr steam and be able to write to it so that output goes to the terminal. Here is the piece of code I have in mind: fd = fileno(stderr); err = fcntl(fd, F_GETFD) if (err != EBADF) { file descriptor is open so I can use stderr as is... } else { stderr is broken we need to rewive it. How? Would I have to create a file descriptor, attach it to terminal and then attach to stderr using fdopen(). Is there anything better? Is there anything that would help me to just output things to the terminal? } I am trying to understand the whole solution and have a consistent approach that deals with all these issues. And one more thing - keeping the file open, I suspect that by default the file should be kept open and we just need to flush it from time to time based on config parameter but there is also a need to allow "append" functionality when file is opened on per event basis. Though this is costly and inefficient to reopen it all the time (no doubts) there might be cases where such functionality is required. I guess I just close and re-open file in this case but async processing would mess things up a bit. You can't just enqueue data to be written and let the event library deal with it. The callback that would write my data to fd would have to complete writing one record and then close fd and open new fd, create a different event and attach next record to it. Something like this. Brain starts to boil a bit when I start thinking about chaining of these callbacks (IMO threads and semaphores are simpler :-)) -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/

2 1

[PATCHES] make enumerations a background task
by Simo Sorce 27 Aug '09

27 Aug '09

This affects only the ldap driver. Enumerations are now a background task, on startup a full enumeration is performed. Then every 5 minutes (changeable default) only new/modified entries are retrieved. So after the first full pull the quantity of data that is pulled is limited, this is why I have not removed the large transaction yet. Next step is to change the group handling code into a 2 step process, or some group membership may be lost against server that allow nested groups via the member/memberof schema. 0001 simplify code, no need for a wrapper 0002 The patch that changes the enumeration code Simo. -- Simo Sorce * Red Hat, Inc * New York

2 3

[PATCHES] remove "legacy" option
by Simo Sorce 27 Aug '09

27 Aug '09

The "legacy" option was introduced to speed up development. It is actually confusing and not necessary. >From now on we store all using the member/memberof schema internally. (Tested only with the native ldap driver). 0001 mainly converts the core sysdb functions 0002 fixes many errors in fill_grent and removes legacy support form there 0003 adds an upgrade function to convert existing databases on the first run. Simo. -- Simo Sorce * Red Hat, Inc * New York

3 4

[PATCH] Initial support for multiple schema types
by Simo Sorce 27 Aug '09

27 Aug '09

While this patch does not add much yet it is a prerequisite for the following ones I am going to send. It sets a paths I want to follow in allowing the ldap driver to handle servers with different schema types (for example use of memberUid vs member/memberOf). Simo. -- Simo Sorce * Red Hat, Inc * New York

2 2

[PATCH] do not show server messages to user
by Sumit Bose 27 Aug '09

27 Aug '09

Hi, with this patch the server messages indicating the reason of an authentication error are not displayed to the user anymore. But they are still send to syslog. Is there anything else which should be sent to syslog according to https://fedorahosted.org/sssd/ticket/32 ? Otherwise I would close the ticket. bye, Sumit

4 5

[PATCH] removed unused header file
by Sumit Bose 27 Aug '09

27 Aug '09

Hi, this patch removes an old and unused header file. bye, Sumit

3 2

[PATCH] Fix usage of wrong structure
by Simo Sorce 26 Aug '09

26 Aug '09

Patch FYI. Pushed under the one-liner rule. -- Simo Sorce * Red Hat, Inc * New York

1 0

[PATCH] Ensure that fds are only added once in the sbus
by Stephen Gallagher 26 Aug '09

26 Aug '09

There were cases that we we were mishandling where D-BUS would try to send us the same file descriptor when changing events. Instead of trying to blindly add a new event (which caused an EEXIST error in epoll), we will remove the old tevent_fd and add the new one. Also, we were trying to be too fancy in the toggle function. It is simpler to just remove or add watches as appropriate. -- Stephen Gallagher RHCE 804006346421761 Looking to carve out IT costs? www.redhat.com/carveoutcosts/

2 1

RFC: Enumerations and sssd drivers
by Simo Sorce 26 Aug '09

26 Aug '09

Hello all, during this month I have been slowly working on a set of patches to move from storing information in 2 different formats (legacy and member/memberOf based) to just one format (member/memberOf based). While doing this I had to address some problems that come up when you want to store a group and its members have not been stored yet, and cases like this. All the while I have been testing doing enumerations against a server that has more than 3k users and 3k groups. This is a medium sized database, and yet getting groups from scratch (startup after deleting the .ldb database) could take up to a minute; granted the operation is quite a bit faster if the database just needs updating and not creation from scratch, but I still think it's too much. I've been thinking hard about how to address this problem and solve the few hacks we have in the code when it comes to enumeration caching and retrieval. We always said that enumerations are evil (and they are indeed) and in fact we even have options that disable enumerations by default. Yet I think this is not necessarily the right way to go. I think we have 2 major problems in our current architecture when it comes to enumerations. 1) we try to hit the wire when an enumeration request comes in from a process and a (small) timeout for the previous enumeration has been expired. 2) We run the enumeration in a single transaction (and yes I have recently introduced this), which means any other operation is blocked until the enumeration is finished. The problem I actually see is that user space apps may have to wait just too much, and this *will* turn out to be a problem. Even if we give the option to turn off enumeration I think that for apps that needs it the penalty has become simply too big. Also I think the way we have to perform updates using this model is largely inefficient, as we basically perform a full new search potentially every few minutes. After some hard thinking I wrote down a few points I'd like the list opinion on. If people agree I will start acting on them. * stop performing enumerations on demand, and perform them in background if enumerations are activated (change the enumeration parameter from a bitfield to a true/flase boolean) * perform a full user+group enumeration at startup (possibly using a paged or vlv search) * when possible request the modifyTimestamp attribute and save the highest modifyTimestamp into the domain entry as originalMaxTimestamp * on a tunable interval run a task that refreshes all users and groups in the background using a search filter that includes &(modifyTimestamp>$originalMaxtimestamp) * still do a full refresh every X minutes/hours * disable using a single huge transaction for enumerations (we might be ok doing a transaction for each page search if pages are small, otherwise just revert to the previous behavior of having a transaction per stored object) * Every time we update an entry we store the originalModifyTimestamp on it as a copy of the remote modifyTimestamp, this allows us to know if we actually need to touch the cached entry at all upon refresh (like when a getpwnam() is called, speeding up operations for entries that need no refresh (we will avoid data transformation and a writing to ldb). * Every time we run the general refresh task or we save a changed entry we store a LastUpdatedTimestamp * When the refresh task is completed successfully we run another cleanup task that searches our LDB for any entry that has a too old LastUpdatedTimestamp. If any is found, we double check against the remote server if the entry still exists (and update it if it does), and otherwise we delete it. NOTE: this means that until the first background enumeration is complete, a getent passwd or a getent group call may return incomplete results. I think this is acceptable as it will really happen only at startup, when the daemon caches are empty. NOTE2: Off course the scheduled refreshes and cleanup tasks are always rescheduled if we are offline or if a fatal error occurs during the task. NOTE3: I am proposing to change only the way enumerations are handled, single user or group lookups will remain unchanged for now and will be dealt with later if needed. Please provide comments or questions if you think there is anything not clear with the proposed items or if you think I forgot to take some important aspect in account. Simo.

3 10

Looking for a bit of advice
by Dmitri Pal 25 Aug '09

25 Aug '09

Hi, I am working on the next set of changes for ELAPI and have a question. With some recent changes to the design the terminology looks like this: target - a destination where the event is sent. Target consists of chain of sinks in priority (fail over) order. So if one sink fails next sink in the chain will be used. Imagine now two sinks in the chain. The first sink is to write to the file on the NFS and the second to write to the local file. They differ in configuration but the code is the same for both sinks. This brings to the notion of "provider". Provider is the implementation (the code) to write to specific destination that can be configured in different ways. The sink then a provider + configuration. Also some of this can be read here: https://fedorahosted.org/sssd/wiki/WikiPage/ELAPIInterface Ok so not when we are clear (hopefully) on the terminology let me ask the question. Under directory that contains mail ELAPI code (which is sssd/common/elapi) I plan to have sub directory named "providers". Under it there will be directory for "file" provider, "syslog" provider, "stderr" provider and other providers. Generally the providers should be implemented as shared libraries and that will be the case. However per Simo's suggestion I plan to embed the three basic providers: "file", "syslog", "stderr" into the ELAPI library itself. This means that these providers have to be built before I build the rest of ELAPI library and its unit tests. Other (loadable) providers do not cause any dependency on the core ELAPI code and can be built separately. So how should I better organize the tree and build process. I can: a) Since the loadable libraries can be built independently they probably should live in other part of the tree, right? Then does it make sense to have the hierarchy that I plan? May I should just put all the code for the basic providers into the elapi directory itself? b) Another option is to keep this code structured as I plan but do not have makefile.am and configure.ac in the subdirectories and build the standard providers from within the elapi makefile. c) Add makefile.am and configure.ac into the "provider" subdirectory and individual provider directories, build independent libraries for each provider and then use these libraries when I build elapi and its unit test. The first option is the simplest but then I will have a mixture of file prefixes in one directory. The source files that constitute core of elapi start with "elapi_" but the providers start with provider name like "file_provider.c", "stderr_provider.c" etc. The second one does not seem to be inline with automake guidelines as far as I understand them. The third one is the most complex. So which way should I go? -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/

1 2

The Monitor returned an error [org.freedesktop.DBus.Error.NoReply]
by Mathias Gug 25 Aug '09

25 Aug '09

Hi, While trying to start sssd 0.5.0 on an Ubuntu Karmic system, I've run into the following error: [sssd[dp]] [id_callback] (0): The Monitor returned an error [org.freedesktop.DBus.Error.NoReply] [sssd] [global_checks_handler] (0): Unknown child (2552) did exit [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector [sssd[be[files]]] [be_cli_init] (0): Failed to connect to monitor services. [sssd[be[files]]] [be_process_init] (0): fatal error setting up server bus [sssd[be[files]]] [main] (0): Could not initialize backend [5] [sssd[pam]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[pam]] [sss_process_init] (0): fatal error setting up backend connector [sssd] [global_checks_handler] (0): Unknown child (2553) did exit [sssd] [global_checks_handler] (0): Unknown child (2554) did exit [sssd] [global_checks_handler] (0): Unknown child (2555) did exit [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector [sssd[pam]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[pam]] [sss_process_init] (0): fatal error setting up backend connector [sssd[be[files]]] [be_cli_init] (0): Failed to connect to monitor services. [sssd[be[files]]] [be_process_init] (0): fatal error setting up server bus [sssd[be[files]]] [main] (0): Could not initialize backend [5] [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector [sssd[pam]] [sss_dp_init] (0): Failed to connect to monitor services. [sssd[pam]] [sss_process_init] (0): fatal error setting up backend connector [...] dbus is installed and running: $ ps -ef | grep dbus 106 2382 1 0 21:27 ? 00:00:00 /bin/dbus-daemon --system The sssd configuration is the example configuration with using the LOCAL domain (/etc/sssd/sssd.conf): [services] description = Local Service Configuration activeServices = nss, dp, pam # Number of times services should attempt to reconnect in the # event of a Data Provider crash or restart before they give up reconnection_retries = 3 [services/nss] description = NSS Responder Configuration # the following prevents sssd for searching for the root user/group in # all domains (you can add here a comma separated list of system accounts are # always going to be /etc/passwd users, or that you want to filter out) filterGroups = root filterUsers = root [services/dp] description = Data Provider Configuration [services/pam] description = PAM Responder Configuration [services/monitor] description = Service Monitor Configuration #if a backend is particularly slow you can raise this timeout here sbusTimeout = 30 [domains] description = Domains served by SSSD domains = LOCAL [domains/LOCAL] description = LOCAL migration domain enumerate = 3 minId = 500 magicPrivateGroups = FALSE legacy = TRUE provider = files -- Mathias Gug Ubuntu Developer http://www.ubuntu.com

3 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

sssd-devel August 2009