[389-users] sub-tree synchronization/watching: persistent search questions

[Petr Spacek]

Hello list,

I would like to get opinions from 389 gurus to following problem.

I have an application (DNS server), which needs to read content of whole one 
sub-tree (cn=dns, dc=test) and keep it synchronized.

The work flow is:
1) Application (DNS server) starts
2) Application reads all existing data out from the sub-tree
3) Application does /something/ with the existing data and starts replying to 
application clients
4) Sub-tree has to be kept in sync with LDAP server, i.e. updates from LDAP 
server should be incrementally applied to the 'state' inside the application


The problem with persistent search is that it doesn't offer any reliable 
'signal' that step (2) ended. The search is just running for infinite time and 
I can't find any signal that all existing entries were read already and now 
the application will get only Entry Change Notifications.


Basically, I'm looking for something like LDAP syncRepl in refreshAndPersist 
mode with no cookie (RFC 4533 section 1.3.2 and section 3.4).


I know that Entry Change Notification from persistent search contains bit 
field which denotes if the entry was added/modified/deleted/nothing (i.e. not 
modified, just read). Unfortunately, this bit field can't be used for 
*reliable* detection that all existing entries were read.


Could this 'hack' work reliably?
1) Start persistent search (in separate application thread), but suspend 
result processing.
2) In another application thread, do the normal sub-tree search on the same 
sub-tree. Normal search will be started *after* the persistent search.
3) Process all results from normal search first
4) Do /something application specific/
5) Start processing updates from persistent search

In my application I can cope with duplicates, when 'normal' search returned 
entry cn=xyz and the persistent search returned the same entry cn=xyz again.


I can see another option:
To implement 389 plugin which will provide (very partial) support for RFC 
4533. The idea is to implement only state-less pieces (no cookies) and return 
some error when client attempts to use a cookie.

Could somebody judge how difficult it can be? From my (naive) point of view 
are state-less parts of RFC 4533 only 'persistent search encapsulated in 
another LDAP controls'.


Thank you very much for your time.

-- 
Petr^2 Spacek