[389-devel] Review of plugin code

[William Brown]

> > 
> > Say for example:
> > 
> > begin
> > add some object Y
> > read Y
> > commit
> > 
> > Does read Y see the object within the transaction?
> 
> Yes.
> 
> > Is there a way to make the
> > search happen so that it occurs outside the transaction, IE it doesn't see 
> > Y?
> 
> Not a nested search operation.  A nested search operation will always 
> use the parent/context transaction.
> 

Makes sense. Just wanted to clarify so that I understood how different apis etc
will behave within a betxn context.


> > 
> > Is there a piece of documentation (perhaps the plugin guide) that lists the
> > order in which these operations are called?
> 
> Not sure, but in general it is:
> 
> incoming operation from client
> front end processing
> preoperation
> call backend
> bepreoperation
> start transaction
> betxnpreoperation
> do operation in the database
> betxnpostoperation
> end transaction
> bepostoperation
> return from backend
> send result to client
> postoperation
> 

I found this list in the Redhat Directory Services Plugin Development guide, but
thank you for it as well.

So with the transaction, there is only one transaction that covers all betxn
plugins? And I also wouldn't need to do anything to start / end the transaction
within a plugin either do I?

> > > The toughest part is the deadlock prevention.  At the start transaction,
> > > it holds a DB lock.  And most plug-ins maintain its own mutex to protect
> > > its resource.  It'd easily cause deadlock situation especially when
> > > multiple plug-ins are enabled (which is common :). So, please be careful
> > > not to acquire/free locks in the wrong order...
> > Of course. This is always an issue in multi-threaded code and anything with
> > locking. Stress tests are probably good to find these deadlocks, no?
> 
> Yes.  There is some code in dblayer.c that will stress the transaction 
> code by locking/unlocking many db pages concurrently with external 
> operations.
> https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
> /dblayer.c#n210
> https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
> /dblayer.c#n4131

Are there some unit tests hooked up that call and test this?

> > That answers some of my question. I guess the larger part of the question is 
> > how
> > the plugin subsystem treats each pluginType differently and the value of 
> > having
> > a plugin register to more than one pluginType. Are there some documents you 
> > can
> > point me to about this?
> 
> I'm not sure if there are docs which answer your specific question. But 
> a plugin may want to perform tasks at different points in the 
> operation.  For example, DNA may want to generate a unique value in the 
> pretxn phase, then roll back that value in the posttxn phase if the 
> operation failed.  Replication wants to many tasks at many different 
> points in the operation.

That makes sense.

Again, I found some documents about this in the plugin developers guide, and
I've re-read some other plugin code so I have a better grasp of this now. 

> 
> > 
> > Additionally, with betxn, this seems quite black-or-white. It's either on a 
> > ds
> > instance that has betxn support, so every update will be betxn capable, or 
> > it's
> > not on such a system so you fall back to other methods. Is this correct? 
> > With
> > new plugins is it even worth writing them without betxn support?
> 
> Correct.  It is not even worth writing a new plugin without betxn 
> support, if it does any update to the database that depends on other 
> updates to the database triggered by the incoming operation from the client.
> 

Fantastic. Thanks for the clarification.


For reference here is the plugin that this message references. It was a simple
demo to just get my head into plugin writing from scratch. I'm happy to accept
comments about it. I know that as a feature it's useless because
nsDS5ReplicaType already covers the use case, but it's a nice simple exercise
none the less.

Thanks for your time.

-- 
William Brown <william at blackhats.net.au>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: read_only.c
Type: text/x-csrc
Size: 5283 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/389-devel/attachments/20150808/b4150077/attachment.bin>