[389-devel] Please review: Nunc-Stans 75, 76

[William Brown]

Hi,

This is a more complex review than normal due to the addition of a new
dependency.

https://pagure.io/nunc-stans/issue/75

https://pagure.io/nunc-stans/issue/76

Related tickets:

https://pagure.io/nunc-stans/issue/67

https://pagure.io/nunc-stans/issue/40

https://pagure.io/nunc-stans/issue/73


I want to propose that we adopt a new library I have created, libsds,
for consumption in nunc-stans and soon directory server. 

https://pagure.io/libsds

libsds contains a single threaded queue, mutex protected queue, lfds
queue wrapper, single thread b+tree, and soon a copy on write b+tree and
lru based on it.

The benefit of this abstraction is:

* reduce the need to add data structures to Directory Server. We have
many custom structures in various modules (bst in valueset, avl for aci,
etc). 
* Testing. The structures in ds are tested only through "production
use". libsds comes with extensive cmocka test suites. 
* Build process, lfds makes the nunc-stans build very complex and
fragile. Libsds wraps this much better, and makes the process very
stable and clean.
* Abstraction. Improvements to libsds will benefit all of DS, rather
than just small parts we have to hand hold at the moment.
* Portability. Libsds supports detection of arch and platform, meaning
when we can not provide a lock free queue, we fall back to the mutex
queue seamlessly.
* Less duplication. The lfds queue requires some handholding to work.
libsds wraps this up. Additionally, encourages us to use these
structures if they are as easy as "init(), push(), pop()". 

As far as Directory Server is concerned, we would bundle and build
libsds just like we currently do for nunc-stans. 

A large benefit of this abstraction is portability, allowing support of
nunc-stans on other platforms like FreeBSD, linux arm64, or other cpus. 

Here is the libsds source:

https://pagure.io/libsds/tree/master

Here is the patch to allow nunc-stans to use this:

https://pagure.io/nunc-stans/issue/raw/files/3f1bb8192449719c74e7c1f1787a...

https://pagure.io/nunc-stans/issue/raw/files/862f38cbd70574adbc78e7f97377...

Finally, the patch to allow nunc-stans on FreeBSD

https://pagure.io/nunc-stans/issue/raw/files/ff0a32398f22b3009f01ed389893...


After this, an extra patch will be needed for Directory Server to
support building a bundle of libsds. I want to use the libsds queue for
the DS logging subsystem soon and the b+tree for replacement of the
conntable in connection.c, so I would really like to see this included.

Thanks,

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane