I've forgotten Linked Attributes plugin, you could also disable it.<br><br>Don't you have some exotic type of index activated for uniqueMember (like substring)? The default value is only the equality index in dse.ldif<br>
<br>dn: cn=uniquemember,cn=default indexes,cn=config,cn=ldbm database,cn=plugins,c<br> n=config<br>objectClass: top<br>objectClass: nsIndex<br>cn: uniquemember<br>nsSystemIndex: false<br>nsIndexType: eq<br><br><br>In any case, batch write loads are quite particular. You could try to play with nsslapd-db-checkpoint-interval and nsslapd-db-durable-transaction config attributes while you run your batch uniqueMember modifications. You could also try disabling completely or limiting logging intensity (<a href="http://directory.fedoraproject.org/wiki/Named_Pipe_Log_Script">http://directory.fedoraproject.org/wiki/Named_Pipe_Log_Script</a>) .<br>
<br>In my VM tests (RHEL5, 1 vCPU Xeon E5640 @ 2.67GHz, 1Gb mem, 389DS v1.2.10.6) with our production environment (~20k users, groups of ~6000 members created as in your case with perl scripts using ldapmodify running on the same VM) a group of 6000 uniqueMembers is created in 3 minutes 10 sec (190s) from scratch. Using "dstat" i see that the main problem is disk writes (transaction logs of db4):<br>
<span style="font-family:courier new,monospace">----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">usr sys idl wai hiq siq| read writ| recv send| in out | int csw </span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 91 0 0 9 0 0| 0 40M| 120B 978B| 0 0 | 316 325 <br> 90 3 0 7 0 0| 0 40M| 60B 310B| 0 0 | 184 294 <br> 92 2 0 5 0 1| 0 40M| 60B 310B| 0 0 | 160 297 <br>
93 1 0 5 0 1| 0 40M| 60B 310B| 0 0 | 159 312 <br> 93 3 1 1 0 2| 0 24M| 60B 310B| 0 0 | 206 372 <br> 76 4 2 18 0 0| 0 40M| 60B 310B| 0 0 | 165 265 <br>
94 0 0 6 0 0| 0 40M| 60B 310B| 0 0 | 221 275 <br> 90 1 0 7 1 1| 0 41M| 60B 310B| 0 0 | 479 313 <br> 86 2 0 11 1 0| 0 40M| 60B 310B| 0 0 | 403 306 <br>
93 0 0 6 0 1| 0 20M| 120B 364B| 0 0 | 489 298 <br> 90 1 0 9 0 0| 0 40M| 60B 310B| 0 0 | 389 296 <br> 88 1 0 11 0 0| 0 40M| 60B 310B| 0 0 | 358 319 <br>
76 0 0 23 1 0| 0 41M| 60B 310B| 0 0 | 403 303 <br><br>@+<br><br></span><br><br><div class="gmail_quote">Le 19 avril 2012 18:50, Russell Beall <span dir="ltr"><<a href="mailto:beall@usc.edu">beall@usc.edu</a>></span> a écrit :<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.<div>
<br></div><div>There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.</div><div><br></div><div>
Regards,</div><div>Russ.</div><div><br><div><div><div class="h5"><div>On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:</div><br></div></div><blockquote type="cite"><div><div class="h5">Hi Russel,<br><br><br><div class="gmail_quote">
Le 18 avril 2012 23:06, Russell Beall <span dir="ltr"><<a href="mailto:beall@usc.edu" target="_blank">beall@usc.edu</a>></span> a écrit :<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div>On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:</div><div><blockquote type="cite"><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium">Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.<br>
</span></blockquote><br></div></div><div><div><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><br>
</span></div></div><div><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium">It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.</span></div>
<div><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><br>
</span></div><div><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium">Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.</span></div>
</div></blockquote><div><br>There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.<br>
<br>@+<br></div></div></div></div><div class="im">
--<br>389 users mailing list<br><a href="mailto:389-users@lists.fedoraproject.org" target="_blank">389-users@lists.fedoraproject.org</a><br><a href="https://admin.fedoraproject.org/mailman/listinfo/389-users" target="_blank">https://admin.fedoraproject.org/mailman/listinfo/389-users</a></div>
</blockquote></div><br></div></div><br>--<br>
389 users mailing list<br>
<a href="mailto:389-users@lists.fedoraproject.org">389-users@lists.fedoraproject.org</a><br>
<a href="https://admin.fedoraproject.org/mailman/listinfo/389-users" target="_blank">https://admin.fedoraproject.org/mailman/listinfo/389-users</a><br></blockquote></div><br>