Re: [389-users] Distributed Numeric Assignment plugin fails with indexing

[Rich Megginson]

On 03/25/2013 10:07 AM, Soeren Malchow wrote:
>  Untitled Document
>
>  Hi Rich,
>
>  we tried the methods described in Nathans post, specifically indexing 
>  the nummer assingned through DNA including the nsMatchingRule, when 
>  indexing, the index processes exactly 1000 entries and 98% and stops 
>  though there are 1017 entries in the database.
>  
Does the index process hang?  Or does it just report 98% and complete 
normally?  You can use dbscan to find out exactly what's in the index.

>  The nsslapd-idlistscanlimit is at the default of 4000 (though we also
>
>  tried increasing it inbetween)
>
>  To
>
>  From dse.ldif
>
>  <- snip->
>
>  nsslapd-idlistscanlimit: 4000
>
>  nsslapd-pagedidlistscanlimit: 0
>
>  <-snip->
>  
That isn't used for creating indexes.
>  The other four databases have much less entries and there is no problem.
>
>  Increasing the loglevel does not give anymore information, I think I 
>  even found the code somewhere the is responsible for the error message 
>  and for me it does not really help.
>
>  The only other numbr that is “1000” is the dnaThreshold but that 
>  should not have anything to do with it, nevertheless I tried 
>  increasing it to 2000 to see if anything changes.
>
>  This is the DNA onfiguration from dse.ldif
>
>  ßsnip à
>
>  dn: cn=organization-employeeNumbers,cn=Distributed Numeric Assignment 
>  Plugin,c
>
>  n=plugins,cn=config
>
>  objectClass: top
>
>  objectClass: extensibleObject
>
>  dnaNextValue: 220380
>
>  dnaMaxValue: 249999
>
>  dnaSharedCfgDN: 
>  cn=organizations-employeeNumbers,ou=Ranges,ou=Organizations,dc
>
>  =XXX,dc=XXX
>
>  dnaThreshold: 2000
>
>  dnaType: employeeNumber
>
>  dnaScope: ou=Organizations,dc=XXX,dc=XXX
>
>  dnaFilter: (objectclass=inetOrgPerson)
>
>  dnaMagicRegen: magic
>  
This is a problem, but not perhaps the only problem.  If employeeNumber 
is INTEGER syntax, "magic" is not a valid value for an integer.  Pick a 
number that cannot possibly be in your range.

>  cn: organization-employeeNumbers
>
>  creatorsName: cn=directory manager
>
>  modifiersName: 
>  uid=admin,ou=administrators,ou=topologymanagement,o=netscaperoo
>
>  t
>
>  createTimestamp: 20120831184744Z
>
>  modifyTimestamp: 20130322022625Z
>
>  ßsnip à
>
>  Unfortunately we are stuck and can’t find anymore to go on.
>
>  I already tried deleting a lot of users (approximately 50 with the DNA 
>  attribute) trying to index again still tells me it indexed 1000 
>  entries (98%)
>
>  And still it is not possible to create users, the Apache Directory 
>  Studio displays the following error
>
>  ßsnip à
>
>  Error while creating entry
>
>  - [LDAP: error code 1 - Allocation of a new value for range 
>  cn=organization-employeenumbers,cn=distributed numeric assignment 
>  plugin,cn=plugins,cn=config failed! Unable to proceed.]
>
>    java.lang.Exception: [LDAP: error code 1 - Allocation of a new value 
>  for range cn=organization-employeenumbers,cn=distributed numeric 
>  assignment plugin,cn=plugins,cn=config failed! Unable to proceed.]
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.checkResponse(DirectoryApiConnectionWrapper.java:1271)
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.access$600(DirectoryApiConnectionWrapper.java:110)
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper$6.run(DirectoryApiConnectionWrapper.java:926)
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.runAndMonitor(DirectoryApiConnectionWrapper.java:1173)
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.checkConnectionAndRunAndMonitor(DirectoryApiConnectionWrapper.java:1107)
>
>                 at 
>
> org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.createEntry(DirectoryApiConnectionWrapper.java:948)
>
>                 at 
>
> org.apache.directory.studio.ldapbrowser.core.jobs.CreateEntryRunnable.createEntry(CreateEntryRunnable.java:224)
>
>                 at 
>
> org.apache.directory.studio.ldapbrowser.core.jobs.CreateEntryRunnable.run(CreateEntryRunnable.java:124)
>
>                 at 
>
> org.apache.directory.studio.connection.ui.RunnableContextRunner$1.run(RunnableContextRunner.java:113)
>
>                 at 
>  org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:121)
>
>    [LDAP: error code 1 - Allocation of a new value for range 
>  cn=organization-employeenumbers,cn=distributed numeric assignment 
>  plugin,cn=plugins,cn=config failed! Unable to proceed.]
>
>  ßsnip à
>
>  Any hint how to pinpoint the problem
>  
Can you post the excerpt from the directory server access and errors log 
from around the time of the apache ds error showing the corresponding 
389 ds errors?

>  Thanks
>
>  Soeren
>
>  *From:*Rich Megginson [mailto:[e-mail]]
>  *Sent:* Montag, 25. März 2013 23:24
>  *To:* Scott Crooks
>  *Cc:* General discussion list for the 389 Directory server project.; 
>  Soeren Malchow
>  *Subject:* Re: [389-users] Distributed Numeric Assignment plugin fails 
>  with indexing
>
>  On 03/24/2013 10:08 PM, Scott Crooks wrote:
>
>      Rich,
>
>      Here's the information you requested.
>
>      *389-ds-base-1.2.11.15-12.el6_4.x86_64* is the version, from the
>      EPEL 6 repositories.
>
>      To be more specific, the *indexes* for that particular database
>      (ou=Organizations,dc=test,dc=net) are exactly at 1000,
>
>
>  I'm still not sure what you mean by 1000 - where does this number come 
>  from?
>
>  Nathan's post had some good information for getting more information 
>  about DNA.
>
>
>      but the database has 1017 entries. There are 953 users
>      (inetOrgPerson's) in the tree (ou=Organizations,dc=test,dc=net)
>      that also have the employeeNumber attribute that are managed by DNA.
>
>      The database for//ou=Organizations,dc=test,dc=net is actually
>      comprised of 4 databases. It's set-up like this:
>
>      1. Company 1: o=Company1,ou=Organizations,dc=test,dc=net has it's
>      own database
>      2. Company 2: o=Company2,ou=Organizations,dc=test,dc=net has it's
>      own database
>      3. Company 3: o=Company3,ou=Organizations,dc=test,dc=net has it's
>      own database
>      4. All other companies fall within the
>      *ou=Organizations,dc=test,dc=net* database.
>
>      The database mentioned above that has 1017 entries is just
>      the*ou=Organizations,dc=test,dc=net* one.
>
>      If you want I can provide exact numbers for the others.
>
>      Best Regards
>
>      Scott Crooks
>      Systems Engineer
>
>      scott.crooks(a)mcon.net <mailto:[e-mail]>
>      Phone: [phone]
>      Mobile: [phone]
>      http://www.mcon.net
>
>      Logo
>
>      	
>
>      MCON China Ltd.
>      Suite 1202, e-Tower
>      12C Guanghua Road
>      Chaoyang District
>      Beijing 100020
>
>      	
>
>      *埃蒙坎信息系**统集****成技**术**(**北****京**)**有限公司**
>      *中国北京市朝阳区光华路丙12号
>      数码01大厦1202室
>      邮编：100020
>
>      Member of mcon group: Australia • Austria • China • Czech Republic
>      • Germany • India • Japan • Malaysia • Russia • South Korea •
>      Switzerland • USA
>
>      On 03/22/2013 11:57 PM, Rich Megginson wrote:
>
>          On 03/22/2013 03:24 AM, Scott Crooks wrote:
>
>              Greetings,
>
>              We're using 389-ds on CentOS 6.4
>
>
>          What  version of 389-ds-base? rpm -q 389-ds-base
>
>
>              with 3 master LDAP servers in different locations. All
>              three master servers have a problem adding new users in
>              our organization database using the DNA plugin.
>
>              We receive the following (unhelpful) error message in the
>              log files:
>
>              /[22/Mar/2013:09:54:37 +0800] NSUniqueAttr - ADD begin
>              [22/Mar/2013:09:54:37 +0800] NSUniqueAttr - ADD parameter
>              untagged: uid
>              [22/Mar/2013:09:54:37 +0800] NSUniqueAttr - ADD
>              target=cn=testuser01,ou=Organizations,dc=test,dc=net
>              *[22/Mar/2013:09:54:37 +0800] dna-plugin - dna_pre_op:
>              failed to allocate a new ID!!
>              [22/Mar/2013:09:54:37 +0800] dna-plugin - dna_pre_op:
>              operation failure [1]
>              *[22/Mar/2013:09:54:37 +0800] roles-plugin - --> roles_post_op
>              [22/Mar/2013:09:54:37 +0800] roles-plugin - -->
>              roles_cache_change_notify
>              [22/Mar/2013:09:54:37 +0800] roles-plugin - <-- roles_post_op/
>
>              We've tried indexing the uidNumber and employeeNumber
>              attributes as described here:
>              *http://www.spinics.net/linux/fedora/389-users/msg10796.html*
>
>              One curious fact: there are exactly 1000 users in our
>              database for ou=Organizations,dc=test,dc=net. When
>              following the instructions for the above link, the log
>              outputs the following
>
>              */organizations: Indexing attribute: employeenumber
>              organizations: Indexed 1000 entries (97%).
>              organizations: Finished indexing./*/
>              /
>              Our organization uses employeenumber rather than
>              uidNumber, but they're the same in the end. We find it
>              strange that the user creation fails at *exactly* 1000
>              entries.
>
>
>          What do you mean "user creation fails at exactly 1000
>          entries"?  Do you mean "after adding the 1000th user entry"? 
>          The error above is from attempting to add cn=testuser01 - was
>          that the 1000th user entry?
>
>
>
>              Any ideas on where the configuration is wrong? We assume
>              it's a configuration problem.
>
>              Best Regards
>
>              Scott Crooks
>              Systems Engineer
>
>              scott.crooks(a)mcon.net <mailto:[e-mail]>
>              Phone: [phone]
>              Mobile: [phone]
>              http://www.mcon.net
>
>              Logo
>
>              	
>
>              MCON China Ltd.
>              Suite 1202, e-Tower
>              12C Guanghua Road
>              Chaoyang District
>              Beijing 100020
>
>              	
>
>              *埃蒙坎信息系**统集****成技**术**(**北****京**)**有限公司**
>              *中国北京市朝阳区光华路丙12号
>              数码01大厦1202室
>              邮编：100020
>
>              Member of mcon group: Australia • Austria • China • Czech
>              Republic • Germany • India • Japan • Malaysia • Russia •
>              South Korea • Switzerland • USA
>
>
>
>
>              --
>
>              389 users mailing list
>
>              389-users(a)lists.fedoraproject.org 
> <mailto:[e-mail]>
>
>              https://admin.fedoraproject.org/mailman/listinfo/389-users
>