Master index (?!) feature request?

[Diego Woitasen]

I didn't know how to title this mail. I think this should be a feature
request in Track when I want to discuss this here first.

I have 389DS with 150 DBs with an structure similar to this:

dc=company,dc=com
ou=Headquarters,dc=company,dc=com
ou=Branch1,dc=company,dc=com
ou=Branch2,dc=company,dc=com
.
.
.
ou=Branch150,dc=company,dc=com

Each one of this subtrees are in separate DBs because I have subtree
replication between the 150 branches of the companies.

80% of the objects are in the ou=HeadQuarters. I've noticed that the
performance is definetely better when I use base ou=Headquarters in my
applications.

I have indexes on each DB but I think that the problem is that 389DS
doesn't have a master index or something to improve the searchs in
scenarios like mine.

May be the solution is to implemen another replication code that
doesn't required separate DBs for subtree replication.

Shall I file a ticket? Or there is a solution now?

Regards,
 Diego

-- 
Diego Woitasen

[Rich Megginson]

On 05/04/2012 06:47 AM, Diego Woitasen wrote:
>  I didn't know how to title this mail. I think this should be a
> feature
>  request in Track when I want to discuss this here first.
>
>  I have 389DS with 150 DBs with an structure similar to this:
>
>  dc=company,dc=com
>  ou=Headquarters,dc=company,dc=com
>  ou=Branch1,dc=company,dc=com
>  ou=Branch2,dc=company,dc=com
>  .
>  .
>  .
>  ou=Branch150,dc=company,dc=com
>
>  Each one of this subtrees are in separate DBs because I have subtree
>  replication between the 150 branches of the companies.
>
>  80% of the objects are in the ou=HeadQuarters. I've noticed that the
>  performance is definetely better when I use base ou=Headquarters in my
>  applications.
>
>  I have indexes on each DB but I think that the problem is that 389DS
>  doesn't have a master index or something to improve the searchs in
>  scenarios like mine. 
Can you explain more about what you mean by "master index"?

>  May be the solution is to implemen another replication code that
>  doesn't required separate DBs for subtree replication.
>
>  Shall I file a ticket? Or there is a solution now?
>
>  Regards,
>    Diego
>

[Diego Woitasen]

On Fri, May 4, 2012 at 10:24 AM, Rich Megginson <rmeggins(a)redhat.com&gt; wrote:
>  On 05/04/2012 06:47 AM, Diego Woitasen wrote:
> >
> > I didn't know how to title this mail. I think this should be a feature
> > request in Track when I want to discuss this here first.
> >
> > I have 389DS with 150 DBs with an structure similar to this:
> >
> > dc=company,dc=com
> > ou=Headquarters,dc=company,dc=com
> > ou=Branch1,dc=company,dc=com
> > ou=Branch2,dc=company,dc=com
> > .
> > .
> > .
> > ou=Branch150,dc=company,dc=com
> >
> > Each one of this subtrees are in separate DBs because I have subtree
> > replication between the 150 branches of the companies.
> >
> > 80% of the objects are in the ou=HeadQuarters. I've noticed that the
> > performance is definetely better when I use base ou=Headquarters in my
> > applications.
> >
> > I have indexes on each DB but I think that the problem is that 389DS
> > doesn't have a master index or something to improve the searchs in
> > scenarios like mine.
>
>
>  Can you explain more about what you mean by "master index"? 
An index that includes all the DBs. May be "global index" is a better
name. Right now, when you search for something, 389DS queries all the
DBs, one by one and with 150 DBs is a problem. There should be a way
to avoid that.


> >
> > May be the solution is to implemen another replication code that
> > doesn't required separate DBs for subtree replication.
> >
> > Shall I file a ticket? Or there is a solution now?
> >
> > Regards,
> >  Diego
> >
>  


-- 
Diego Woitasen

[Rich Megginson]

On 05/04/2012 07:44 AM, Diego Woitasen wrote:
>  On Fri, May 4, 2012 at 10:24 AM, Rich
> Megginson<rmeggins(a)redhat.com>  wrote:
> > On 05/04/2012 06:47 AM, Diego Woitasen wrote:
> >> I didn't know how to title this mail. I think this should be a feature
> >> request in Track when I want to discuss this here first.
> >>
> >> I have 389DS with 150 DBs with an structure similar to this:
> >>
> >> dc=company,dc=com
> >> ou=Headquarters,dc=company,dc=com
> >> ou=Branch1,dc=company,dc=com
> >> ou=Branch2,dc=company,dc=com
> >> .
> >> .
> >> .
> >> ou=Branch150,dc=company,dc=com
> >>
> >> Each one of this subtrees are in separate DBs because I have subtree
> >> replication between the 150 branches of the companies.
> >>
> >> 80% of the objects are in the ou=HeadQuarters. I've noticed that the
> >> performance is definetely better when I use base ou=Headquarters in my
> >> applications.
> >>
> >> I have indexes on each DB but I think that the problem is that 389DS
> >> doesn't have a master index or something to improve the searchs in
> >> scenarios like mine.
> >
> > Can you explain more about what you mean by "master index"?
>  An index that includes all the DBs. May be "global index" is a better
>  name. Right now, when you search for something, 389DS queries all the
>  DBs, one by one and with 150 DBs is a problem. There should be a way
>  to avoid that. 
Ok, I see.  Yes, might be useful too for doing simple paged searches, 
server side sorting, vlv, etc. across multiple databases.

> >
> >> May be the solution is to implemen another replication code that
> >> doesn't required separate DBs for subtree replication.
> >>
> >> Shall I file a ticket? Or there is a solution now?
> >>
> >> Regards,
> >>   Diego
> >>
>
>

[Mark Reynolds]

Deigo,

In the meantime, you should get a performance boost if your top "tree" 
suffix(dc=company,dc=com) has the same attributes indexed as all the 
other sub-suffixes(db's).  Even if the db is empty, this will still help 
when you search on the top node.

Mark

On 05/04/2012 10:15 AM, Rich Megginson wrote:
>  On 05/04/2012 07:44 AM, Diego Woitasen wrote:
> > On Fri, May 4, 2012 at 10:24 AM, Rich Megginson<rmeggins(a)redhat.com>  
> > wrote:
> >> On 05/04/2012 06:47 AM, Diego Woitasen wrote:
> >>> I didn't know how to title this mail. I think this should be a feature
> >>> request in Track when I want to discuss this here first.
> >>>
> >>> I have 389DS with 150 DBs with an structure similar to this:
> >>>
> >>> dc=company,dc=com
> >>> ou=Headquarters,dc=company,dc=com
> >>> ou=Branch1,dc=company,dc=com
> >>> ou=Branch2,dc=company,dc=com
> >>> .
> >>> .
> >>> .
> >>> ou=Branch150,dc=company,dc=com
> >>>
> >>> Each one of this subtrees are in separate DBs because I have subtree
> >>> replication between the 150 branches of the companies.
> >>>
> >>> 80% of the objects are in the ou=HeadQuarters. I've noticed that the
> >>> performance is definetely better when I use base ou=Headquarters in my
> >>> applications.
> >>>
> >>> I have indexes on each DB but I think that the problem is that 389DS
> >>> doesn't have a master index or something to improve the searchs in
> >>> scenarios like mine.
> >>
> >> Can you explain more about what you mean by "master index"?
> > An index that includes all the DBs. May be "global index" is a better
> > name. Right now, when you search for something, 389DS queries all the
> > DBs, one by one and with 150 DBs is a problem. There should be a way
> > to avoid that.
>
>  Ok, I see.  Yes, might be useful too for doing simple paged searches, 
>  server side sorting, vlv, etc. across multiple databases.
>
> >
> >
> >>
> >>> May be the solution is to implemen another replication code that
> >>> doesn't required separate DBs for subtree replication.
> >>>
> >>> Shall I file a ticket? Or there is a solution now?
> >>>
> >>> Regards,
> >>>   Diego
> >>>
> >
> >
>
>  -- 
>  389 users mailing list
>  389-users(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/389-users

[Diego Woitasen]

On Fri, May 4, 2012 at 3:57 PM, Mark Reynolds <mareynol(a)redhat.com&gt; wrote:
>  Deigo,
>
>  In the meantime, you should get a performance boost if your top "tree"
>  suffix(dc=company,dc=com) has the same attributes indexed as all the other
>  sub-suffixes(db's).  Even if the db is empty, this will still help when you
>  search on the top node.
>
>  Mark
>
>
>  On 05/04/2012 10:15 AM, Rich Megginson wrote:
> >
> > On 05/04/2012 07:44 AM, Diego Woitasen wrote:
> >>
> >> On Fri, May 4, 2012 at 10:24 AM, Rich Megginson&lt;rmeggins(a)redhat.com&gt;
> >>  wrote:
> >>>
> >>> On 05/04/2012 06:47 AM, Diego Woitasen wrote:
> >>>>
> >>>> I didn't know how to title this mail. I think this should be a
> feature
> >>>> request in Track when I want to discuss this here first.
> >>>>
> >>>> I have 389DS with 150 DBs with an structure similar to this:
> >>>>
> >>>> dc=company,dc=com
> >>>> ou=Headquarters,dc=company,dc=com
> >>>> ou=Branch1,dc=company,dc=com
> >>>> ou=Branch2,dc=company,dc=com
> >>>> .
> >>>> .
> >>>> .
> >>>> ou=Branch150,dc=company,dc=com
> >>>>
> >>>> Each one of this subtrees are in separate DBs because I have subtree
> >>>> replication between the 150 branches of the companies.
> >>>>
> >>>> 80% of the objects are in the ou=HeadQuarters. I've noticed that the
> >>>> performance is definetely better when I use base ou=Headquarters in my
> >>>> applications.
> >>>>
> >>>> I have indexes on each DB but I think that the problem is that 389DS
> >>>> doesn't have a master index or something to improve the searchs in
> >>>> scenarios like mine.
> >>>
> >>>
> >>> Can you explain more about what you mean by "master index"?
> >>
> >> An index that includes all the DBs. May be "global index" is a better
> >> name. Right now, when you search for something, 389DS queries all the
> >> DBs, one by one and with 150 DBs is a problem. There should be a way
> >> to avoid that.
> >
> >
> > Ok, I see.  Yes, might be useful too for doing simple paged searches,
> > server side sorting, vlv, etc. across multiple databases.
> >
> >>
> >>
> >>>
> >>>> May be the solution is to implemen another replication code that
> >>>> doesn't required separate DBs for subtree replication.
> >>>>
> >>>> Shall I file a ticket? Or there is a solution now?
> >>>>
> >>>> Regards,
> >>>>  Diego
> >>>>
> >>
> >>
> >
> > --
> > 389 users mailing list
> > 389-users(a)lists.fedoraproject.org
> > https://admin.fedoraproject.org/mailman/listinfo/389-users 
I've filed a ticket https://fedorahosted.org/389/ticket/357

-- 
Diego Woitasen