389-devel August 2015

389-devel@lists.fedoraproject.org

8 participants
38 discussions

Instance discovery tools in lib389
by William Brown 13 Aug '15

13 Aug '15

Hi, I have been working on lib389 recently to bring in a number of the tools I have developed professionally into the project so that others can benefit from this. As you may have seen I've already submitted a few patches related to this. I've done some of the api changes needed, but now I would like to make it a bit easier to add command line tools that can utilise lib389. I would like to add a "tools" directory to lib389 with a number of the cli tools and helpers that I have written [1]. Is this the best location for these tools? Some of them could arguably go into the main line 389ds codebase, but I want to use lib389 to back them. Second, with these tools I don't want them to be complex. For example, I would like a tool that can show what attribute can be held by what objectClass. I would want the calling syntax to be as easy as: attribute_query.py -i <instance name> <attribute name> In order to achieve this there are a few changes I want to make. First, is that there is already a DirSrv.list() function. This works well to list instances on the system, but it's missing the instance name, so I want to add this to the dictionary that is returned. Second, I want to make it possible to initialise the DirSrv object from an instance directory returned by the list() function. Normally this is done through the allocate() function to establish the details of the connection, at the moment the instance function doesn't return enough data to support this. Either, I can add in functionality to the list() function to return more data about each instance, possibly enough to just take the result from list() and put it in as allocate(args) and have a working connection. Alternately I write a new allocate_from_instance() (or some other name ...?) function that is able to do the discovery of the needed data, and then call allocate. Which would be the preferred method. Comments and advice is appreciated. [1] The tools I want to add are: * List all known system instances * schema query tools (list of object classes, attributes, query which objectclasses can take which attribute) * aci testing and manipulation tools. For example, "does aci X do what I suspect". * aci inversion tool. convert != aci's into == aci's to help conversion to white lists. * Probably many more that I haven't thought of yet to help make administration of a 389ds instance simpler. -- William Brown <william(a)blackhats.net.au>

2 8

please review: Ticket 47848 - lib389 - add function to create simple ldif files
by Mark Reynolds 13 Aug '15

13 Aug '15

https://fedorahosted.org/389/ticket/48248 https://fedorahosted.org/389/attachment/ticket/48248/0001-Ticket-47848-Add-…

1 0

please review: Ticket 48247 - lib389 - Change default user from "nobody" to "dirsrv" for instance creation
by Mark Reynolds 13 Aug '15

13 Aug '15

https://fedorahosted.org/389/ticket/48247 https://fedorahosted.org/389/attachment/ticket/48247/0001-Ticket-48247-Use-…

1 0

Review Request for a new ACL testcase
by Simon Pichugin 12 Aug '15

12 Aug '15

Hi, I have automated one testcase for: https://fedorahosted.org/389/ticket/47569 This is my patch: https://fedorahosted.org/389/attachment/ticket/47569/0001-Ticket-47569-Adde… Please, review my change. Kind regards, Simon Pichugin

1 0

Please review: [389 Project] #48245: Man pages and help for remove-ds.pl doesn't display "-a" option
by Noriko Hosoi 11 Aug '15

11 Aug '15

https://fedorahosted.org/389/ticket/48245 https://fedorahosted.org/389/attachment/ticket/48245/0001-Ticket-48245-Man-…

1 0

please review: Ticket 47686 - Removing chaining database link triggers invalid read
by Mark Reynolds 10 Aug '15

10 Aug '15

https://fedorahosted.org/389/ticket/47686 https://fedorahosted.org/389/attachment/ticket/47686/0001-Ticket-47686-remo…

1 0

please review: Ticket 47891 - memberof/retrocl deadlock - fix coverity issues
by Mark Reynolds 10 Aug '15

10 Aug '15

https://fedorahosted.org/389/ticket/47931 https://fedorahosted.org/389/attachment/ticket/47931/0002-Ticket-47931-Fix-…

1 0

Re: [389-devel] Review of plugin code
by William Brown 10 Aug '15

10 Aug '15

On Thu, 2015-08-06 at 14:25 -0700, Noriko Hosoi wrote: > Hi William, > > Very interesting plug-in! Thanks. As a plugin, it's value is quite useless due to the nsDS5ReplicaType flags. But it's a nice simple exercise to get ones head around how the plugin architecture works from scratch. It's one thing to patch a plugin, compared to writing one from nothing. > > Regarding betxn plug-in, it is for putting the entire operation -- the > primary update + associated updates by the enabled plug-ins -- in one > transaction. By doing so, the entire updates are committed to the DB if > and only if all of the updates are successful. Otherwise, all of them > are rolled back. That guarantees there will be no consistency among > entries. Okay, so if I can be a pain, how to betxn handle reads? Do reads come from within the transaction? Or is there a way to read from the database outside the transaction. Say for example: begin add some object Y read Y commit Does read Y see the object within the transaction? Is there a way to make the search happen so that it occurs outside the transaction, IE it doesn't see Y? > > In that sense, your read-only plug-in is not a good example for betxn > since it does not do any updates. :) Considering the purpose of the > "read-only" plug-in, invoking it at the pre-op timing (before the > transaction) would be the best. Very true! I kind of knew what betxn did, but I wanted to confirm more completely in my mind. So I think what my read-only plugin does at the moment works quite nicely then outside of betxn. Is there a piece of documentation (perhaps the plugin guide) that lists the order in which these operations are called? > > Since MEP requires the updates on the DB, it's supposed to be called in > betxn. That way, what was done in the MEP plug-in is committed or > rolled back together with the primary updates. Makes sense. > > The toughest part is the deadlock prevention. At the start transaction, > it holds a DB lock. And most plug-ins maintain its own mutex to protect > its resource. It'd easily cause deadlock situation especially when > multiple plug-ins are enabled (which is common :). So, please be careful > not to acquire/free locks in the wrong order... Of course. This is always an issue in multi-threaded code and anything with locking. Stress tests are probably good to find these deadlocks, no? > > About your commented out code in read_only.c, I guess you copied the > part from mep.c and are wondering what it is for? > > There are various type of plug-ins. > > $ egrep nsslapd-pluginType dse.ldif | sort | uniq > nsslapd-pluginType: accesscontrol > nsslapd-pluginType: bepreoperation > nsslapd-pluginType: betxnpostoperation > nsslapd-pluginType: betxnpreoperation > nsslapd-pluginType: database > nsslapd-pluginType: extendedop > nsslapd-pluginType: internalpreoperation > nsslapd-pluginType: matchingRule > nsslapd-pluginType: object > nsslapd-pluginType: preoperation > nsslapd-pluginType: pwdstoragescheme > nsslapd-pluginType: reverpwdstoragescheme > nsslapd-pluginType: syntax > > The reason why slapi_register_plugin and slapi_register_plugin_ext were > implemented was: > > /* > * Allows a plugin to register a plugin. > * This was added so that 'object' plugins could register all > * the plugin interfaces that it supports. > */ > > On the other hand, MEP has this type. > > nsslapd-pluginType: betxnpreoperation > > The type is not "object", but the MEP plug-in is implemented as having > the type. Originally, it might have been "object"... Then, we > introduced the support for "betxn". To make the transition to "betxn" > smoothly, we put the code to check "betxn" is in the type. If there is > "betxn" as in "betxnpreoperation", call the plug-in in betxn, otherwise > call them outside of the transaction. Having the switch in the > configuration, we could go back to the original position without > rebuilding the plug-in. > > Since we do not go back to pre-betxn era, the switch may not be too > important. But keeping it would be a good idea for the consistency with > the other plug-ins. > > Does this answer you question? Please feel free to let us know if it > does not. That answers some of my question. I guess the larger part of the question is how the plugin subsystem treats each pluginType differently and the value of having a plugin register to more than one pluginType. Are there some documents you can point me to about this? Additionally, with betxn, this seems quite black-or-white. It's either on a ds instance that has betxn support, so every update will be betxn capable, or it's not on such a system so you fall back to other methods. Is this correct? With new plugins is it even worth writing them without betxn support? > I'm sure our team is interested in your idea and work, so let me share > your test plug-in with them. Sure. It's not really that useful, like I said, nsDS5ReplicaType already does this job. It was just an exercise to get my head more into the framework before I work on http://directory.fedoraproject.org/docs/389ds/design/mep-rework.html Thanks for the review, and for answering my questions. Your advice has helped a lot! Sincerely,

2 3

Review Request for test cases
by Amita 07 Aug '15

07 Aug '15

Hi All, I have automated few test cases for https://fedorahosted.org/389/attachment/ticket/47910/ . Here is my patch :: https://fedorahosted.org/389/attachment/ticket/47910/0001-Ticket-47910-allo… I request for your valuable feedback. Thanks & Regards, Amita

2 3

please review: Ticket 47703 - Remove search limit for ACI group evaluation
by Mark Reynolds 07 Aug '15

07 Aug '15

https://fedorahosted.org/389/ticket/47703 https://fedorahosted.org/389/attachment/ticket/47703/0001-Ticket-47703-remo…

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-devel August 2015