[389-devel] RFC: New Design: Fine Grained ID List Size

[David Boreham]

On 9/13/2013 2:18 PM, Rich Megginson wrote:
> On 09/12/2013 07:08 PM, David Boreham wrote:
>> On 9/11/2013 11:41 AM, Howard Chu wrote:
>>>
>>> Just out of curiosity, why is keeping a count per key a problem? If 
>>> you're using BDB duplicate key support, can't you just use 
>>> cursor->c_count() to get this? I.e., BDB already maintains key 
>>> counts internally, why not leverage that?
>>>
>>
>> afaik you need to pass the DB_RECNUM flag at DB creation time to get 
>> record counting behavior, and it imposes a performance and 
>> concurrency penalty on writes. Also afaik 389DS does not set that 
>> flag except on VLV indexes (which need it, and coincidentally were 
>> the original reason for the feature being added to BDB).
>
> I'm using bdb 4.7 on RHEL 6.
> Looking at the code, it appears the dbc->count method for btree is 
> __bamc_count() in bt_cursor.c.  I'm not sure, but it looks as though 
> this function has to iterate each page counting the duplicates on each 
> page, which makes it a non-starter.  Unless I'm mistaken, it doesn't 
> look as though it keeps a counter on each update, then simply returns 
> the counter.  I don't see any code which would make the behavior 
> different depending on if DB_RECNUM is used when the database is created.

The DB_RECNUM count feature is not accessed via dbc->count() but through 
the dbc->c_get() call, passing DB_GET_RECNO, positioning at the last 
key. You do also need to use nested btrees for it to count the dups, 
afaik (but we're doing that in the DS indexes already I believe).