[389-devel] Review of plugin code

[Rich Megginson]

On 08/07/2015 05:54 PM, William Brown wrote:
>>> Say for example:
>>>
>>> begin
>>> add some object Y
>>> read Y
>>> commit
>>>
>>> Does read Y see the object within the transaction?
>> Yes.
>>
>>> Is there a way to make the
>>> search happen so that it occurs outside the transaction, IE it doesn't see
>>> Y?
>> Not a nested search operation.  A nested search operation will always
>> use the parent/context transaction.
>>
> Makes sense. Just wanted to clarify so that I understood how different apis etc
> will behave within a betxn context.
>
>
>>> Is there a piece of documentation (perhaps the plugin guide) that lists the
>>> order in which these operations are called?
>> Not sure, but in general it is:
>>
>> incoming operation from client
>> front end processing
>> preoperation
>> call backend
>> bepreoperation
>> start transaction
>> betxnpreoperation
>> do operation in the database
>> betxnpostoperation
>> end transaction
>> bepostoperation
>> return from backend
>> send result to client
>> postoperation
>>
> I found this list in the Redhat Directory Services Plugin Development guide, but
> thank you for it as well.
>
> So with the transaction, there is only one transaction that covers all betxn
> plugins?

Yes and no.  Yes, there is only one parent transaction corresponding to 
the initial database interaction from the original top level incoming 
client operation.  However, each plugin will create a nested transaction 
for its write.  For example, a top level modify operation will create a 
top level parent database transaction, and may cause the memberof plugin 
to trigger and write.  The nested memberof plugin operation will create 
a nested child transaction using the original top level transaction as 
the parent.

Note that all of these transactions are created/managed without any 
knowledge or participation of plugin code - the database code manages 
all of this for the plugins.

> And I also wouldn't need to do anything to start / end the transaction
> within a plugin either do I?

No.

>
>>>> The toughest part is the deadlock prevention.  At the start transaction,
>>>> it holds a DB lock.  And most plug-ins maintain its own mutex to protect
>>>> its resource.  It'd easily cause deadlock situation especially when
>>>> multiple plug-ins are enabled (which is common :). So, please be careful
>>>> not to acquire/free locks in the wrong order...
>>> Of course. This is always an issue in multi-threaded code and anything with
>>> locking. Stress tests are probably good to find these deadlocks, no?
>> Yes.  There is some code in dblayer.c that will stress the transaction
>> code by locking/unlocking many db pages concurrently with external
>> operations.
>> https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
>> /dblayer.c#n210
>> https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
>> /dblayer.c#n4131
> Are there some unit tests hooked up that call and test this?

No, not afaik.

>
>>> That answers some of my question. I guess the larger part of the question is
>>> how
>>> the plugin subsystem treats each pluginType differently and the value of
>>> having
>>> a plugin register to more than one pluginType. Are there some documents you
>>> can
>>> point me to about this?
>> I'm not sure if there are docs which answer your specific question. But
>> a plugin may want to perform tasks at different points in the
>> operation.  For example, DNA may want to generate a unique value in the
>> pretxn phase, then roll back that value in the posttxn phase if the
>> operation failed.  Replication wants to many tasks at many different
>> points in the operation.
> That makes sense.
>
> Again, I found some documents about this in the plugin developers guide, and
> I've re-read some other plugin code so I have a better grasp of this now.
>
>>> Additionally, with betxn, this seems quite black-or-white. It's either on a
>>> ds
>>> instance that has betxn support, so every update will be betxn capable, or
>>> it's
>>> not on such a system so you fall back to other methods. Is this correct?
>>> With
>>> new plugins is it even worth writing them without betxn support?
>> Correct.  It is not even worth writing a new plugin without betxn
>> support, if it does any update to the database that depends on other
>> updates to the database triggered by the incoming operation from the client.
>>
> Fantastic. Thanks for the clarification.
>
>
> For reference here is the plugin that this message references. It was a simple
> demo to just get my head into plugin writing from scratch. I'm happy to accept
> comments about it. I know that as a feature it's useless because
> nsDS5ReplicaType already covers the use case, but it's a nice simple exercise
> none the less.

Sure.  It is a complex area, so questions are expected and encouraged.

>
> Thanks for your time.
>
>
>
> --
> 389-devel mailing list
> 389-devel at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/389-devel/attachments/20150810/011fba4d/attachment.html>