On 08/07/2015 05:54 PM, William Brown wrote:

      

        

          
Say for example:

begin
add some object Y
read Y
commit

Does read Y see the object within the transaction?


        

        
Yes.



        

          
Is there a way to make the
search happen so that it occurs outside the transaction, IE it doesn't see 
Y?


        

        
Not a nested search operation.  A nested search operation will always 
use the parent/context transaction.



      

      
Makes sense. Just wanted to clarify so that I understood how different apis etc
will behave within a betxn context.




      

        

          
Is there a piece of documentation (perhaps the plugin guide) that lists the
order in which these operations are called?


        

        
Not sure, but in general it is:

incoming operation from client
front end processing
preoperation
call backend
bepreoperation
start transaction
betxnpreoperation
do operation in the database
betxnpostoperation
end transaction
bepostoperation
return from backend
send result to client
postoperation



      

      
I found this list in the Redhat Directory Services Plugin Development guide, but
thank you for it as well.

So with the transaction, there is only one transaction that covers all betxn
plugins?

    

    

    Yes and no.  Yes, there is only one parent transaction corresponding
    to the initial database interaction from the original top level
    incoming client operation.  However, each plugin will create a
    nested transaction for its write.  For example, a top level modify
    operation will create a top level parent database transaction, and
    may cause the memberof plugin to trigger and write.  The nested
    memberof plugin operation will create a nested child transaction
    using the original top level transaction as the parent.

    

    Note that all of these transactions are created/managed without any
    knowledge or participation of plugin code - the database code
    manages all of this for the plugins.

    

    

      
And I also wouldn't need to do anything to start / end the transaction
within a plugin either do I?

    

    

    No.

    

    

      



      

        

          

            
The toughest part is the deadlock prevention.  At the start transaction,
it holds a DB lock.  And most plug-ins maintain its own mutex to protect
its resource.  It'd easily cause deadlock situation especially when
multiple plug-ins are enabled (which is common :). So, please be careful
not to acquire/free locks in the wrong order...


          

          
Of course. This is always an issue in multi-threaded code and anything with
locking. Stress tests are probably good to find these deadlocks, no?


        

        
Yes.  There is some code in dblayer.c that will stress the transaction 
code by locking/unlocking many db pages concurrently with external 
operations.
https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
/dblayer.c#n210
https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/servers/slapd/back-ldbm
/dblayer.c#n4131


      

      
Are there some unit tests hooked up that call and test this?

    

    

    No, not afaik.

    

    

      



      

        

          
That answers some of my question. I guess the larger part of the question is 
how
the plugin subsystem treats each pluginType differently and the value of 
having
a plugin register to more than one pluginType. Are there some documents you 
can
point me to about this?


        

        
I'm not sure if there are docs which answer your specific question. But 
a plugin may want to perform tasks at different points in the 
operation.  For example, DNA may want to generate a unique value in the 
pretxn phase, then roll back that value in the posttxn phase if the 
operation failed.  Replication wants to many tasks at many different 
points in the operation.


      

      
That makes sense.

Again, I found some documents about this in the plugin developers guide, and
I've re-read some other plugin code so I have a better grasp of this now. 



      

        

        

          
Additionally, with betxn, this seems quite black-or-white. It's either on a 
ds
instance that has betxn support, so every update will be betxn capable, or 
it's
not on such a system so you fall back to other methods. Is this correct? 
With
new plugins is it even worth writing them without betxn support?


        

        
Correct.  It is not even worth writing a new plugin without betxn 
support, if it does any update to the database that depends on other 
updates to the database triggered by the incoming operation from the client.



      

      
Fantastic. Thanks for the clarification.


For reference here is the plugin that this message references. It was a simple
demo to just get my head into plugin writing from scratch. I'm happy to accept
comments about it. I know that as a feature it's useless because
nsDS5ReplicaType already covers the use case, but it's a nice simple exercise
none the less.

    

    

    Sure.  It is a complex area, so questions are expected and
    encouraged.

    

    

      

Thanks for your time.



      

      

      

      
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel