lib389 usage cheatsheet

[Simon Pichugin]

Hi team,
recently, we had a discussion on a scrum meeting about lib389 and its new API. 

If I understood right there was an opinion that lib389 DSLdapObjects API
is not very intuitive and it is much easier stick to python-ldap style
because it uses ldapmodify/ldapadd wording (or close enough to it).
And I partially agree with it (and I already have some thoughts how we can
improve it).

Many patches on the review show that the people don't change their code
to the new way.
I've given some thought to it and this is what I came up with:

1. I think it is okay to use instance.add_s and instance.modify_s
   for simple operations.
2. If you'd like to make your life easier, you can use DSLdapObjects API
   (and I'll help you with that)
3. We should stop using old lib389 API because we don't support it anymore
   and it will be depricated in the future. We should use DSLdapObject
   instead and we should improve it if we don't like something about it.

We have a lot of docstrings written in lib389 code and the code itself
is pretty readable, in my opinion. But I'd like to make the life easier
and I've started to write 'lib389 cheatsheet'. It has old way/new way
comparison blocks and I base it on the real-life code from your patches
adding some commentairies.

For now, it is here:
https://fedorapeople.org/~spichugi/html/cheatsheet.html

Thanks!
Simon

[thierry bordaz]

Hi Simon,

    Thanks Simon starting this thread :)

    Currently lib389 is mostly used by ldap devel/QE and seems realistic
    to become the admin library (for example used by freeipa or others)
    and component of 389 administrative tools. LDAP knowledge is a
    requirement for all of them. In that sense, lib389 should continue
    to follow/offer basic LDAP elements (connections, naming_ctx,
    req/resp, synchronous/asynchronous, extop, control...).
    I agree that those elements are a bit frustrating for people wanting
    to use LDAP as a simple repository. The DSLdapObject abrstaction
    looks very promising. It can be oriented to simple use case: create
    users and groups, manage memberships, authentication/rights,
    automatic deployement on several replicated instances. Then extended
    on demand.
    In short, IMHO I would prefer to keep the most of the LDAP elements
    (option 1) and propose/extend easy POC interface (option 2). I am
    not sure the work DSLdapObject is my favorite, we could name it
    according to the use case we want to propose.
    For me option 3 will be the worse option. I remember an abstraction
    layer that had a so high level that I constantly looked at the
    access/error log, to be sure that the program was doing what I was
    thinking it should.

    best regards
    thierry


On 05/09/2018 04:56 PM, Simon Pichugin wrote:
>  Hi team,
>  recently, we had a discussion on a scrum meeting about lib389 and its new API.
>
>  If I understood right there was an opinion that lib389 DSLdapObjects API
>  is not very intuitive and it is much easier stick to python-ldap style
>  because it uses ldapmodify/ldapadd wording (or close enough to it).
>  And I partially agree with it (and I already have some thoughts how we can
>  improve it).
>
>  Many patches on the review show that the people don't change their code
>  to the new way.
>  I've given some thought to it and this is what I came up with:
>
>  1. I think it is okay to use instance.add_s and instance.modify_s
>      for simple operations.
>  2. If you'd like to make your life easier, you can use DSLdapObjects API
>      (and I'll help you with that)
>  3. We should stop using old lib389 API because we don't support it anymore
>      and it will be depricated in the future. We should use DSLdapObject
>      instead and we should improve it if we don't like something about it.
>
>  We have a lot of docstrings written in lib389 code and the code itself
>  is pretty readable, in my opinion. But I'd like to make the life easier
>  and I've started to write 'lib389 cheatsheet'. It has old way/new way
>  comparison blocks and I base it on the real-life code from your patches
>  adding some commentairies.
>
>  For now, it is here:
>  https://fedorapeople.org/~spichugi/html/cheatsheet.html
>
>  Thanks!
>  Simon
>  _______________________________________________
>  389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org

[Simon Pichugin]

Hi Thierry,
I agree with you, it was exactly my proposition :)

Keeping main python-ldap elements is important because we don't want
to implement or mask/wrap this basic functionality (like working with controls, etc)
we just want to redirect them.

Ideally, we should make our admin library very intuitive for the people
who already knows python-ldap and just LDAP concepts.

I will continue adding more information to the docs regarding DSLdapObject
and I'll try to make it a bit closer to LDAP concepts (at least in
wording), I already have something in mind.

Thanks,
Simon


On Wed, May 09, 2018 at 06:12:33PM +0200, thierry bordaz wrote:
>    Hi Simon,
> 
>      Thanks Simon starting this thread :)
>      Currently lib389 is mostly used by ldap devel/QE and seems realistic
>      to become the admin library (for example used by freeipa or others)
>      and component of 389 administrative tools. LDAP knowledge is a
>      requirement for all of them. In that sense, lib389 should continue
>      to follow/offer basic LDAP elements (connections, naming_ctx,
>      req/resp, synchronous/asynchronous, extop, control...).
>      I agree that those elements are a bit frustrating for people wanting
>      to use LDAP as a simple repository. The DSLdapObject abrstaction
>      looks very promising. It can be oriented to simple use case: create
>      users and groups, manage memberships, authentication/rights,
>      automatic deployement on several replicated instances. Then extended
>      on demand.
>      In short, IMHO I would prefer to keep the most of the LDAP elements
>      (option 1) and propose/extend easy POC interface (option 2). I am
>      not sure the work DSLdapObject is my favorite, we could name it
>      according to the use case we want to propose.
>      For me option 3 will be the worse option. I remember an abstraction
>      layer that had a so high level that I constantly looked at the
>      access/error log, to be sure that the program was doing what I was
>      thinking it should.
>      best regards
>      thierry
>

[William Brown]

On Fri, 2018-05-11 at 09:41 +0200, Simon Pichugin wrote:
>  Hi Thierry,
>  I agree with you, it was exactly my proposition :)
>  
>  Keeping main python-ldap elements is important because we don't want
>  to implement or mask/wrap this basic functionality (like working with
>  controls, etc)
>  we just want to redirect them.
>  
>  Ideally, we should make our admin library very intuitive for the
>  people
>  who already knows python-ldap and just LDAP concepts. 
The issue is that if we want to use python-ldap, we wouldn't have
lib389. We'd just use raw ldap all over the place .... 

>  
>  I will continue adding more information to the docs regarding
>  DSLdapObject
>  and I'll try to make it a bit closer to LDAP concepts (at least in
>  wording), I already have something in mind.
>  
>  Thanks,
>  Simon 
>  1. I think it is okay to use instance.add_s and instance.modify_s
>     for simple operations. 
The issue here is that Entry() isn't py3 safe. It also really limits
what we can do, and the whole "dirsrv is a subclass of
simpleldapobject" and intercepts operations to create Entry is just
complex.

>  2. If you'd like to make your life easier, you can use
> DSLdapObjects
>  API
>     (and I'll help you with that)
>  3. We should stop using old lib389 API because we don't support it
>  anymore
>     and it will be depricated in the future. We should use
>  DSLdapObject
>     instead and we should improve it if we don't like something about
>  it. 

The point of DSLdapObject is to:

* capture knowledge of administration to an accessible api
* To make a bridge between py2/3
* to give a higher level abstraction that "raw ldap" (which is frankly
unusable)
* to allow the elimination of Entry() and the DirSrv subclassing of
simple ldap object.

So everytime you find your self writing ".add_s()" or ".modify_s()",
it's REALLY LIKELY that an adminsitrator needs to do this too!

And although it's more work to encapsulate this to dsldapobject, it
means we have implemented it once and can re-use it - and the jump from
dsldapobject to a cli tool is very very short. It also makes the new
webui easier because that will need to access all those apis too!

There is much more vision here in this api then maybe I let on. It's a
much bigger scope than some convinence functions, it's actually a
gateway to ldap administration to eliminate the unusability of ldap.

Of course, given my history with lib389, I would strongly advise us to
follow options 2 or 3. I'm also happy to write up and knowledge
transfer anything needed for the team. (And I apologise deeply for my
recent abscence on mailing lists, I was traveling, and needed some time
off).

Thanks,

William,

[Viktor Ashirov]

On Mon, May 14 2018 at 13:16:02 +1000, William Brown <william(a)blackhats.net.au&gt;
wrote:
>  On Fri, 2018-05-11 at 09:41 +0200, Simon Pichugin wrote:
> > Hi Thierry,
> > I agree with you, it was exactly my proposition :)
> > 
> > Keeping main python-ldap elements is important because we don't want
> > to implement or mask/wrap this basic functionality (like working with
> > controls, etc)
> > we just want to redirect them.
> > 
> > Ideally, we should make our admin library very intuitive for the
> > people
> > who already knows python-ldap and just LDAP concepts.
>
>  The issue is that if we want to use python-ldap, we wouldn't have
>  lib389. We'd just use raw ldap all over the place .... 
>
> > 
> > I will continue adding more information to the docs regarding
> > DSLdapObject
> > and I'll try to make it a bit closer to LDAP concepts (at least in
> > wording), I already have something in mind.
> > 
> > Thanks,
> > Simon
>
> > 1. I think it is okay to use instance.add_s and instance.modify_s
> >    for simple operations.
>
>  The issue here is that Entry() isn't py3 safe. It also really limits
>  what we can do, and the whole "dirsrv is a subclass of
>  simpleldapobject" and intercepts operations to create Entry is just
>  complex.
>
> > 2. If you'd like to make your life easier, you can use DSLdapObjects
> > API
> >    (and I'll help you with that)
> > 3. We should stop using old lib389 API because we don't support it
> > anymore
> >    and it will be depricated in the future. We should use
> > DSLdapObject
> >    instead and we should improve it if we don't like something about
> > it.
>
>
>  The point of DSLdapObject is to:
>
>  * capture knowledge of administration to an accessible api
>  * To make a bridge between py2/3
>  * to give a higher level abstraction that "raw ldap" (which is frankly
>  unusable)
>  * to allow the elimination of Entry() and the DirSrv subclassing of
>  simple ldap object.
>
>  So everytime you find your self writing ".add_s()" or ".modify_s()",
>  it's REALLY LIKELY that an adminsitrator needs to do this too! 
It's not
just about CLI tools. Please don't forget about tests. There we
need fine-grained access to LDAP primitives to write correct
reproducers. High level abstractions can stand in a way and do not
*exactly* do what is required.
>  And although it's more work to encapsulate this to dsldapobject, it
>  means we have implemented it once and can re-use it - and the jump from
>  dsldapobject to a cli tool is very very short. It also makes the new
>  webui easier because that will need to access all those apis too!
>
>  There is much more vision here in this api then maybe I let on. It's a
>  much bigger scope than some convinence functions, it's actually a
>  gateway to ldap administration to eliminate the unusability of ldap.
>
>  Of course, given my history with lib389, I would strongly advise us to
>  follow options 2 or 3. I'm also happy to write up and knowledge
>  transfer anything needed for the team. (And I apologise deeply for my
>  recent abscence on mailing lists, I was traveling, and needed some time
>  off).
>
>  Thanks,
>
>  William,
>
>  _______________________________________________
>  389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org

[William Brown]

>  On 14 May 2018, at 18:26, Viktor Ashirov <vashirov(a)redhat.com&gt;
> wrote:
>  
> > On Mon, May 14 2018 at 13:16:02 +1000, William Brown <william(a)blackhats.net.au&gt;
> wrote:
> >> On Fri, 2018-05-11 at 09:41 +0200, Simon Pichugin wrote:
> >> Hi Thierry,
> >> I agree with you, it was exactly my proposition :)
> >> 
> >> Keeping main python-ldap elements is important because we don't want
> >> to implement or mask/wrap this basic functionality (like working with
> >> controls, etc)
> >> we just want to redirect them.
> >> 
> >> Ideally, we should make our admin library very intuitive for the
> >> people
> >> who already knows python-ldap and just LDAP concepts.
> > 
> > The issue is that if we want to use python-ldap, we wouldn't have
> > lib389. We'd just use raw ldap all over the place .... 
> > 
> >> 
> >> I will continue adding more information to the docs regarding
> >> DSLdapObject
> >> and I'll try to make it a bit closer to LDAP concepts (at least in
> >> wording), I already have something in mind.
> >> 
> >> Thanks,
> >> Simon
> > 
> >> 1. I think it is okay to use instance.add_s and instance.modify_s
> >>   for simple operations.
> > 
> > The issue here is that Entry() isn't py3 safe. It also really limits
> > what we can do, and the whole "dirsrv is a subclass of
> > simpleldapobject" and intercepts operations to create Entry is just
> > complex.
> > 
> >> 2. If you'd like to make your life easier, you can use DSLdapObjects
> >> API
> >>   (and I'll help you with that)
> >> 3. We should stop using old lib389 API because we don't support it
> >> anymore
> >>   and it will be depricated in the future. We should use
> >> DSLdapObject
> >>   instead and we should improve it if we don't like something about
> >> it.
> > 
> > 
> > The point of DSLdapObject is to:
> > 
> > * capture knowledge of administration to an accessible api
> > * To make a bridge between py2/3
> > * to give a higher level abstraction that "raw ldap" (which is frankly
> > unusable)
> > * to allow the elimination of Entry() and the DirSrv subclassing of
> > simple ldap object.
> > 
> > So everytime you find your self writing ".add_s()" or
> ".modify_s()",
> > it's REALLY LIKELY that an adminsitrator needs to do this too!
>  It's not just about CLI tools. Please don't forget about tests. There we
>  need fine-grained access to LDAP primitives to write correct
>  reproducers. High level abstractions can stand in a way and do not
>  *exactly* do what is required. 
The high level abstraction is not that high level once you look at it. There are plenty of
ways to do low level actions though the dsldapobjects api, it’s meant to be able to test
everything and retain that control.

Regardless if you arent comfortable with that, then use python ldap directly. Entry in lib
389 is a bad abstraction that causes more issues than it solves, and should be removed. 


> > 
> > And although it's more work to encapsulate this to dsldapobject, it
> > means we have implemented it once and can re-use it - and the jump from
> > dsldapobject to a cli tool is very very short. It also makes the new
> > webui easier because that will need to access all those apis too!
> > 
> > There is much more vision here in this api then maybe I let on. It's a
> > much bigger scope than some convinence functions, it's actually a
> > gateway to ldap administration to eliminate the unusability of ldap.
> > 
> > Of course, given my history with lib389, I would strongly advise us to
> > follow options 2 or 3. I'm also happy to write up and knowledge
> > transfer anything needed for the team. (And I apologise deeply for my
> > recent abscence on mailing lists, I was traveling, and needed some time
> > off).
> > 
> > Thanks,
> > 
> > William,
> > 
> > _______________________________________________
> > 389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org