[389-users] 389-console LDIF export question

Rich Megginson rmeggins at redhat.com
Fri Jun 28 14:32:44 UTC 2013


On 06/28/2013 02:14 AM, Michael Lang wrote:
> On 06/27/2013 08:52 PM, Rich Megginson wrote:
>> On 06/27/2013 12:32 PM, Michael Lang wrote:
>>> On 06/27/2013 06:13 PM, Rich Megginson wrote:
>>>> On 06/27/2013 04:57 AM, Michael Lang wrote:
>>>>> Dear all,
>>>>>
>>>>> I would like to clarify how the procedure to "export -> LDIF" 
>>>>> through the 389-console GUI is done (or how if reproduce should be 
>>>>> done).
>>>>> As far as I've understood the exports on the server itself is done 
>>>>> through creating a appropriate entry in cn=export,cn=task,cn=config
>>>>> For "remote" machines using the GUI this isn't valid as the 
>>>>> nsFilename attribute requires a local write able location. What 
>>>>> I've seen through looking at the
>>>>> LDAP protocol queries done during an export-to-ldif on console 
>>>>> machine task is a query to the schema listing all attributeTypes 
>>>>> which are then sent as attribute-retrieve-list in the ldap search. 
>>>>> At least that's how I was able to reproduce the same LDIF from 
>>>>> GUI-Console export and manually. Is this correct and should this 
>>>>> be done in that way ? (I've noticed also the "Warning: If you 
>>>>> don't have permissions" statement when using the LDIF export in 
>>>>> the Console which more-or-less ack's my approach)
>>>>
>>>> Right.  There is no way to do a database export to LDIF to a remote 
>>>> file on a remote machine.
>>>>
>>>> Are you trying to figure out a way to generate an LDIF file using 
>>>> ldapsearch that looks exactly like a database export LDIF file? Why?
>>>
>>> reason for that, I might not have "file based" access on the server 
>>> itself but need to do backup's of the database which are not "setup 
>>> specific" (with setup specific I mean without the right database 
>>> configurations I might not be able to import the content).
>>>
>>> in this special case why I started to look into the ldif create, I 
>>> had database backups but ran into a situation where my directory 
>>> console was showing content but ldapsearch (and other 
>>> tools/applications) where not getting any content at all. It was a 
>>> strange situation and restoring the database trough restore didn't 
>>> help where droping the databases and importing and LDIF was working 
>>> (I still had another server running with the content). As mentioned 
>>> above for the machines I have under complete access this shouldn't 
>>> be the issue, but I also have other machines where I might be forced 
>>> to do this through the LDAP protocol.
>>
>> That particular case sounds like you were missing the "aci" attribute 
>> which is an operational attribute.
>>
>> In general, if you want "everything"(1), you can do a search like this:
>>
>> ldapsearch -b dc=your,dc=suffix 
>> '(|(objectclass=*)(objectclass=ldapsubentry))' \* $OPERATIONAL_ATTRS
>>
>> where $OPERATIONAL_ATTRS is a list of operational attributes you can 
>> get from the schema.  You'll definitely want "aci" in that list.
>>
>> If you know python, python-ldap has a nice schema parser.
>>
>> (1) This still won't get you everything - specifically, you won't get 
>> deleted entries and other replication metadata - If you think you 
>> need that information, add (objectclass=nsTombstone) to the search 
>> filter above.
>
>
> Rich,
>
> thanks for you explanation, the 389-console GUI itself doesn't export 
> nsTombstone objects and the purpose is to be able to restore the data 
> not the replication setting for the database the suffice resides on 
> (at least that's what I would understood from the attributes in the 
> entry returned by the search)
>
> again, thanks for the python-schema hint, much easier then doing it 
> manually, this is how I would scripted reproduce the 389-console LDIF 
> Database Export without the GUI)
>
> <source lang="python">
> #!/usr/bin/env python
> import ldap
> import ldap.schema
> import logging
> import sys
> import ldif
>
> logger  = logging.getLogger('LDIFexport')
> logger.addHandler(logging.StreamHandler(sys.stderr))
> logger.setLevel(logging.INFO)
>
> ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
>
> class LDIFDumper(object):
>     def __init__(self, uri=None):
>         self.uri                = uri
>         self.dn, self.schema    = ldap.schema.urlfetch(self.uri)
>         self.objclass           = set()
>         self.attributes         = set()
>         self._objects           = []
>         self.__fetch_objectClasses__()
>         self.__fetch_attributes__()
>     def __fetch_objectClasses__(self):
>         for obj in self.schema.tree(ldap.schema.ObjectClass).keys():
>             obj = self.schema.get_obj(ldap.schema.ObjectClass, obj)
>             if obj == None:
>                 continue
>             self.objclass.update(obj.names)
>     def __fetch_attributes__(self):
>         for attr in self.schema.tree(ldap.schema.AttributeType).keys():
>             attr = self.schema.get_obj(ldap.schema.AttributeType, attr)
>             if attr == None:
>                 continue
>             self.attributes.update(attr.names)
>     def db2ldif(self, basedn=None, binddn=None, bindpw=None, 
> fileName=None):
>         try:
>             if not isinstance(fileName, ldif.LDIFWriter):
>                 if not isinstance(fileName, file):
>                     fileName    = open(fileName, 'w')
>                 ldifFile    = ldif.LDIFWriter(fileName)
>         except IOError, e:
>             return (False, u'unable to create LDIFFile %s' % 
> fileName.name)
>         try:
>             self.srv    = ldap.initialize(self.uri)
>             self.srv.start_tls_s()
>         except ldap.CONNECT_ERROR:
>             logger.info(u'unable to initialize start_tls, continue 
> without encryption')
>         except ldap.SERVER_DOWN:
>             return (False, u'cannot connect to %s SERVER_DOWN' % 
> self.uri)
>         try:
>             if not binddn == None:
>                 if self.srv.simple_bind_s(binddn, bindpw) != (97, []):
>                     return (False, u'couldn\'t authenticate as %s 
> against %s' % (binddn, self.srv._uri))
>         except ldap.INVALID_CREDENTIALS:
>             return (False, u'couldn\'t authenticate as %s against %s' 
> % (binddn, self.srv._uri))
>         try:
>             # for additional data change filterstr to
>             # 
> filterstr='(|(|(objectClass=*)(objectClass=ldapsubentry))(objectClass=nsTombstone))' 
> by Rich Megginson <rmeggins at redhat.com>
>             for dn, attrs in self.srv.search_s(basedn, 
> ldap.SCOPE_SUBTREE, attrlist=list(self.attributes)):
>                 self._objects.append((dn, attrs))
>                 ldifFile.unparse(dn, attrs)
>         except ldap.ADMINLIMIT_EXCEEDED:
>             return (False, u'couldn\'t fetch all records as %s need 
> more privileges or increase limits on the server' % binddn)
>         except ldap.SIZELIMIT_EXCEEDED:
>             return (False, u'couldn\'t fetch all records as %s need 
> more privileges or increase limits on the server' % binddn)
>         self.srv.unbind_s()
>         return (True, True)
>
> if __name__ == '__main__':
>     import getpass
>     lfd = LDIFDumper(uri='ldap://localhost/')
>     lfd.db2ldif(basedn='dc=ctbto,dc=org', binddn='cn=Directory 
> Manager', bindpw=getpass.getpass(), fileName='/tmp/dc=ctbto,dc=org.ldif')
>
> </source>
>
> head from /tmp/dc=ctbto,dc=org.ldif
>
> dn: dc=ctbto,dc=org
> aci: (targetattr != "userPassword") (version 3.0; acl "Anonymous 
> access"; al
>  low (read, search, compare)userdn = "ldap:///anyone";)
> aci: (targetattr != "nsroledn||aci")(version 3.0; acl "Allow self 
> entry modi
>  fication except for nsroledn and aci attributes"; allow (write)userdn 
> ="lda
>  p:///self";)
> aci: (targetattr = "*")(version 3.0; acl "Configuration Adminstrator"; 
> allow
>   (all) userdn = 
> "ldap:///uid=admin,ou=Administrators,ou=TopologyManagement,
>  o=NetscapeRoot";)
> aci: (targetattr ="*")(version 3.0;acl "Configuration Administrators 
> Group";
>  allow (all) (groupdn = "ldap:///cn=Configuration Administrators, 
> ou=Groups,
>   ou=TopologyManagement, o=NetscapeRoot");)
> aci: (targetattr = "*")(version 3.0; acl "SIE Group"; allow 
> (all)groupdn = "
>  ldap:///dc=ctbto,dc=org";)
> createTimestamp: 20130626125103Z
> creatorsName: cn=directory manager
> domaincomponent: ctbto
> entrydn: dc=ctbto,dc=org
> entryid: 1
> hasSubordinates: FALSE
> modifiersName: cn=directory manager
> modifyTimestamp: 20130626125103Z
> nsUniqueId: f13af0a6-de5e11e2-ab7e96d1-4e21dbc8
> numSubordinates: 0
> objectClass: top
> objectClass: dcobject
> subschemaSubentry: cn=schema
>

I just thought of another wrinkle - a db export does not include virtual 
attributes (such as nsRole, or attributes provided by Class of 
Service).  You will want to exclude those.  To do this, you will need to 
add the LDAP Control LDAP_CONTROL_REAL_ATTRS_ONLY (OID 
2.16.840.1.113730.3.4.17) to your search request.

>
> this should be sufficient in the sense of being able to restore the 
> content of the Directory independent of the "original database 
> distribution". Comparing the object attributes from the manually fetch 
> one and the GUI fetched on there are on both objects 19 attributes. Of 
> course it doesn't provide any configuration (cn=config) or replication 
> settings but that's not the main goal of this exercise.
>
> thanks for you support

>
> regards
> mIke
>
>>>
>>> thanks for your response
>>>
>>> regards
>>> mIke
>>>
>>>
>>>>
>>>>>
>>>>> thanks for any hint.
>>>>> regards
>>>>>
>>>>> Mike
>>>>>
>>>>> -- 
>>>>> 389 users mailing list
>>>>> 389-users at lists.fedoraproject.org
>>>>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>>>>
>>>
>>
>




More information about the 389-users mailing list