Hi,
This is split into two parts.
https://fedorahosted.org/389/ticket/48820
First, is a set of python 3 fixes:
https://fedorahosted.org/389/attachment/ticket/48820/0001-Ticket-Fixes-fo...
Second is the new lib389 api I would like us to move towards.
https://fedorahosted.org/389/attachment/ticket/48820/0002-Ticket-48820-Pr...
This also cleans up and moves some other cli tools around (But I can break that out if we
want). I included it because I adapted
some of the tools to use the new style.
I the best examples are in backend.py, config.py. The parent classes come from
_mapped_object.py
-- Why would you write this?
Lets look at lib389 right now. We have an api that is very inconsistent. Some have
get/set, some getProperties/setProperties.
Some take a single value, some take a key, some take an array. Some are based on passing
an identifer the function to act on,
some are just singletons. Each one is attached to DirSrv, and makes a monolithic and huge
api.
In the end, it looks like lib389 is annoying to work with, and our own team has resorted
to calling inst.add_s/modify_s on
values even under cn=config: and we have a inst.config.get/set already!
-- What does the new api look like?
The idea is that all of our configurations in cn=config derive two styles:
First, a static once off object, with a known basedn and properties. (ie cn=config)
The second is a subtree of objects each with the same attribute types, but that implement
many instances. (IE backends)
This new way of handling this manages both types. The idea is to share and re-use as much
as possible.
Lets have a look at config.py. This is now a subclass of DSLdapObject.
class Config(DSLdapObject):
....
def __init__(self, conn, batch=False):
super(Config, self).__init__(instance=conn, batch=batch)
self._dn = DN_CONFIG
Our super type, defines these methods:
def __unicode__(self):
def __str__(self):
def set(self, key, value):
def get(self, key):
def remove(self, key):
def delete(self):
So we can now, just by deriving the type get access to string printing:
config = Config(instance)
print(config)
>> 'cn=config'
We get a set / get on keys
config.set('nsslapd-accesslog-level', '1')
r = config.get('nsslapd-rootdn')
Remove will remove a value for the attr, and delete will delete the object (unless you set
self._protected, then it cannot be
deleted. This is the default)
A future idea is the batch flag. This will make it so that:
config.set('k', 'v')
config.set('x', 'y')
config.commit() <<-- This actually does the ldap mod of k, x at once.
This lets us make lots of changes in a more efficient way, and some values need to be
updated in sync.
And all we had to do was override self._basedn in class Config()! and we picked up so many
functions straight out.
So lets have a look at the backends now.
The backends (note the plural) is derived from DSLdapObjects
class Backends(DSLdapObjects):
def __init__(self, instance, batch=False):
super(Backends, self).__init__(instance=instance, batch=False)
self._objectclasses = [BACKEND_OBJECTCLASS_VALUE]
self._create_objectclasses = self._objectclasses + ['top',
'extensibleObject' ]
self._filterattrs = ['cn', 'nsslapd-suffix',
'nsslapd-directory']
self._basedn = DN_LDBM
self._childobject = Backend
self._rdn_attribute = 'cn'
self._must_attributes = ['nsslapd-suffix', 'cn']
Here we define that our "Backends" all assert certain properties.
* They can be found with a certain objectclass
* they must be created with a set of classes.
* We can uniquely identify them based on the filterattributes.
* They are all found under some basedn
* They are named off the "cn" attribute
* They must contain a cn and a nsslapd-suffix
The _childobject type defines what the *single* backend instance is.
Because of the inheritence, Backends already gains:
def list(self):
def get(self, selector):
def create(self, rdn=None, properties=None):
Additionally, there is an internal method (that will be explained below)
def _validate(self, rdn, properties):
Just from setting the attributes of the class, we can now list all backends on the
system:
bes = Backends(inst)
print(bes.list())
We can select a backend based on one of the values of a matching attribute in
_filterattrs
be = bes.get('userRoot')
be = bes.get('dc=example,dc=com')
be = bes.get('/var/lib/dirsrv/slapd-localhost/db/userRoot')
The be instance we get back in the Backend type. This is derive from DSLdapObject: Just
like our config. This means it has all
the same methods, such as get, set, __unicode__, as our config! We can see how little code
it takes to do this:
class Backend(DSLdapObject):
def __init__(self, instance, dn=None, batch=False):
super(Backend, self).__init__(instance, dn, batch)
self._naming_attr = 'cn'
That is the *entire* definition of Backend.
Finally, the true power of DSLdapObjects is when we go to *create* a new instance.
Creation of objects is something that in lib389 is hard. We do a lot of validation and
checking to be sure of some things.
Because we are deriving this type, we can already do a baseline of validation in our
creation.
The pattern in:
bes = Backends(inst)
be = bes.create(properties={'nsslapd-suffix': suffix, 'cn':
'userRoot'})
That's it.
The reason for the _validate method on Backends is it allows us to hook and do custom
validation for the type. In this case,
backends can also take the lib389 properties style dictionary, and _validate will re-map
the attributes correctly:
be = bes.create(properties={'suffix': suffix, 'name':
'userRoot'})
_validate will help us by checking:
* Is properties a valid dictionary of types?
* Do we have a valid rdn (from self._rdn_attribute)
* Do we have all the values of self._must_attributes satisfied?
* Is our rdn going to be utf-8 (which python 3 expects?)
At first look it seems like it could be a complex api. But you consider the needed work to
extend and create say:
class RSAEncrption(DSLdapObject):
self.__init__(self, instance=None, batch=False):
super(RSAEncryption, self).__init__(instance, batch)
self._basedn = 'cn=RSA,cn=encryption,cn=config'
And that's it. No more having to write modify, add, etc. We can easily, quickly, and
confidently map our Directory Server
configuration types into lib389. In the future with rest389 this will pay itself off
massively, as a consistent, clean, reliable
api is going to make creation and deployment of a rest admin console much, much more
effecient.
-- Isn't this going to end up nearly being a complete rewrite.
Yes. But it needs to happen. Lib389 is straining, and hard to edit right now. We should
improve this.
-- But aren't there risks here of breaking all our tests?
Yes. But there is a solution.
In backend.py youll note I have:
class BackendLegacy(object):
This is the *original* Backend type that DirSrv attaches too. It's still accesible:
def __add_brookers__(self):
...
from lib389.backend import BackendLegacy as Backend
This way, we can rename our existing types to <NAME>Legacy, and still use them in
tests. I will add a "deprecation" flag to
them, and if we decide to accept the new style of api I will begin to not only re-write
our existing types, but our tests that
rely on them.
This way we can stage the transition over time.
-- Are there any other wins here?
Yes. I basically finished the port of lib389 to python3 in the process of this.
-- Does that mean we can merge this without breaking our existing code and tests?
Yes it does!
--
Sincerely,
William Brown
Software Engineer
Red Hat, Brisbane