autosizing the cache

[Sergei Gerasenko]

Hello,

My cn=userRoot,cn=ldbm database,cn=plugins,cn=config is currently:

...
nsslapd-cachesize: -1
nsslapd-cachememsize: 1543503872
nsslapd-readonly: off
nsslapd-require-index: off
nsslapd-dncachememsize: 500000000
…

But cn=config,cn=ldbm database,cn=plugins,cn=config has these settings:
...
nsslapd-cache-autosize: 10
nsslapd-cache-autosize-split: 40
…


Do I understand correctly that if I remove nsslapd-cachememsize and nsslapd-dncachememsize
from cn=userRoot, the caches will be auto sized? In other words, they are preventing the
autosizing?

Thanks!
 Sergei

[Mark Reynolds]

On 03/01/2018 02:32 PM, Sergei Gerasenko wrote:
>  Hello,
>
>  My cn=userRoot,cn=ldbm database,cn=plugins,cn=config is currently:
>
>  ...
>  nsslapd-cachesize: -1
>  nsslapd-cachememsize: 1543503872
>  nsslapd-readonly: off
>  nsslapd-require-index: off
>  nsslapd-dncachememsize: 500000000
>  …
>
>  But cn=config,cn=ldbm database,cn=plugins,cn=config has these settings:
>  ...
>  nsslapd-cache-autosize: 10
>  nsslapd-cache-autosize-split: 40
>  …
>
>
>  Do I understand correctly that if I remove nsslapd-cachememsize and
> nsslapd-dncachememsize from cn=userRoot, the caches will be auto sized? In other words,
> they are preventing the autosizing? 
Correct, set them to 0 for autotuning to take
effect

http://www.port389.org/docs/389ds/design/autotuning.html
>  Thanks!
>   Sergei
>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org

[Sergei Gerasenko]

Cool. The default setup of 389-ds (version 1.3.5.10) I don’t see either
nsslapd-cache-autosize or nsslapd-cache-autosize-split. Should I just add them to the dse
file?

>  Correct, set them to 0 for autotuning to take effect 
The Redhat docs are a bit confusing on this
(https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
<https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...>):

>  nsslapd-cache-autosize
>  This settings controls if auto-sizing is enabled for the database and entry cache.
> Auto-sizing is enabled:
>  For both the database and entry cache, if the nsslapd-cache-autosize parameter is set to
> a value greater than 0.
>  For the database cache, if the nsslapd-cache-autosize and nsslapd-dbcachesize parameters
> are set to 0.
>  For the entry cache, if the nsslapd-cache-autosize and nsslapd-cachememsizeparameters are
> set to 0. 
You just confirmed bullet 2 and 3. But bullet 1 is not clear: if I
setnsslapd-cache-autosize to something greater than 0 and both types of caches become
auto-tuned, why would then I need to set them to 0 (to enable auto-tuning) individually?

Also, is there a way to check that auto-tuning is working normally? Is dbmon.sh the right
way?

Thanks again,
 Sergei

[Mark Reynolds]

On 03/01/2018 03:18 PM, Sergei Gerasenko wrote:
>  Cool. The default setup of 389-ds (version 1.3.5.10) I don’t see
>  either |nsslapd-cache-autosize or ||nsslapd-cache-autosize-split.
>  Should I just add them to the dse file?| 
|I don't believe autotuning exists in
1.3.5, it was only added to 1.3.6
- sorry :-/|
> > Correct, set them to 0 for autotuning to take effect
>
>  The Redhat docs are a bit confusing on this
>
> (https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...):
Ugh, that is confusing.  I'll file a doc bug on that....
> > |nsslapd-cache-autosize|
> >     This settings controls if auto-sizing is enabled for the database
> >     and entry cache. Auto-sizing is enabled:
> >
> >      *
> >         For both the database and entry cache, if
> >         the |nsslapd-cache-autosize| parameter is set to a value
> >         greater than |0|.
> >      *
> >         For the database cache, if
> >         the |nsslapd-cache-autosize| and |nsslapd-dbcachesize| parameters
> >         are set to |0|.
> >      *
> >         For the entry cache, if
> >         the |nsslapd-cache-autosize| and |nsslapd-cachememsize|parameters
> >         are set to |0|.
> >
>  You just confirmed bullet 2 and 3. But bullet 1 is not clear: if I
>  set|nsslapd-cache-autosize to something greater than 0 and both types
>  of caches become auto-tuned, why would then I need to set them to 0
>  (to enable auto-tuning) individually?|
>  |
>  |
>  |Also, is there a way to check that auto-tuning is working normally?
>  Is dbmon.sh the right way?| 
|The error log at startup will tell you what the server
sets the caches
to.  ldapsearch on nsslapd-dbcachesize will also return the adjusted
size if I am correct, but I haven't tried it.  But like I said before...
this feature is not present in 389-ds-base-1.3.5

Regards,
Mark
|
>  |
>  |
>  |Thanks again,|
>  | Sergei|

[Sergei Gerasenko]

>  I don't believe autotuning exists in 1.3.5, it was only added to
> 1.3.6 - sorry :-/ 
Ah, that makes my life a bit simpler for now :)

> > Also, is there a way to check that auto-tuning is working
> normally? Is dbmon.sh the right way?
>  The error log at startup will tell you what the server sets the caches to.  ldapsearch on
> nsslapd-dbcachesize will also return the adjusted size if I am correct, but I haven't
> tried it.  But like I said before... this feature is not present in 389-ds-base-1.3.5
Got it. Thanks, Mark!!

[Sergei Gerasenko]

Sorry to resurrect this thread, but I’ve upgraded to 1.3.6 and so my question is back on
the table :)

My cn=userRoot,cn=ldbm database,cn=plugins,cn=config is currently:

...
nsslapd-cachesize: -1
nsslapd-cachememsize: 1543503872
nsslapd-readonly: off
nsslapd-require-index: off
nsslapd-dncachememsize: 500000000
…

But cn=config,cn=ldbm database,cn=plugins,cn=config has these settings:
...
nsslapd-cache-autosize: 10
nsslapd-cache-autosize-split: 40
…

If you remember, according to the docs, setting nsslapd-cache-autosize to a value > 0,
enables autosizing for all backends regardless of their cache size settings. Is that
accurate or must I set them to 0s after all? It looks like autosizing is enforced because
I got an error when trying to modify the changelog backend:

modifying entry "cn=changelog,cn=ldbm database,cn=plugins,cn=config"
ldap_modify: Server is unwilling to perform (53)
        additional info: Error: "nsslapd-cachememsize" can not be updated while
"nsslapd-cache-autosize" is set in "cn=config,cn=ldbm
database,cn=plugins,cn=config".

Thanks!

>  On Mar 1, 2018, at 2:55 PM, Sergei Gerasenko
> <gerases(a)gmail.com&gt; wrote:
>  
> > I don't believe autotuning exists in 1.3.5, it was only added to 1.3.6 - sorry
> :-/
>  
>  Ah, that makes my life a bit simpler for now :)
>  
> >> Also, is there a way to check that auto-tuning is working normally? Is dbmon.sh
> the right way?
> > The error log at startup will tell you what the server sets the caches to. 
> ldapsearch on nsslapd-dbcachesize will also return the adjusted size if I am correct, but
> I haven't tried it.  But like I said before... this feature is not present in
> 389-ds-base-1.3.5
>  
>  Got it. Thanks, Mark!!

[William Brown]

On Tue, 2018-03-13 at 13:05 -0500, Sergei Gerasenko wrote:
>  Sorry to resurrect this thread, but I’ve upgraded to 1.3.6 and so my
>  question is back on the table :)
>  
>  My cn=userRoot,cn=ldbm database,cn=plugins,cn=config is currently:
>  
>  ...
>  nsslapd-cachesize: -1
>  nsslapd-cachememsize: 1543503872
>  nsslapd-readonly: off
>  nsslapd-require-index: off
>  nsslapd-dncachememsize: 500000000
>  …
>  
>  But cn=config,cn=ldbm database,cn=plugins,cn=config has these
>  settings:
>  ...
>  nsslapd-cache-autosize: 10
>  nsslapd-cache-autosize-split: 40
>  …
>  
>  If you remember, according to the docs, setting nsslapd-cache-
>  autosize to a value > 0, enables autosizing for all backends
>  regardless of their cache size settings. Is that accurate or must I
>  set them to 0s after all? It looks like autosizing is enforced
>  because I got an error when trying to modify the changelog backend:
>  
>  modifying entry "cn=changelog,cn=ldbm database,cn=plugins,cn=config"
>  ldap_modify: Server is unwilling to perform (53)
>          additional info: Error: "nsslapd-cachememsize" can not be
>  updated while "nsslapd-cache-autosize" is set in "cn=config,cn=ldbm
>  database,cn=plugins,cn=config". 
If autosize > 0, we write a new entrychace/cachememsize every start up.

So you only need to set autosize to between 1 and 99 for it to work.

There is some other logic in there to account for other scenarioes =
for example, if you set autosize to 0 AND you set the entry cachesize
to 0, we'll autosize it at start up anyway. BUT if you have autosize =
0, and entry cachesize > 0, we won't touch it.

Does that help? 

>  
>  Thanks!
>  
>  > On Mar 1, 2018, at 2:55 PM, Sergei Gerasenko <gerases(a)gmail.com&gt;
>  > wrote:
>  > 
>  > > I don't believe autotuning exists in 1.3.5, it was only added to
>  > > 1.3.6 - sorry :-/
>  > 
>  > Ah, that makes my life a bit simpler for now :)
>  > 
>  > > > Also, is there a way to check that auto-tuning is working
>  > > > normally? Is dbmon.sh the right way?
>  > > The error log at startup will tell you what the server sets the
>  > > caches to.  ldapsearch on nsslapd-dbcachesize will also return
>  > > the adjusted size if I am correct, but I haven't tried it.  But
>  > > like I said before... this feature is not present in 389-ds-base-
>  > > 1.3.5
>  > 
>  > Got it. Thanks, Mark!!
>  
>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
>  rg 
-- 
Thanks,

William Brown

[Sergei Gerasenko]

Hi William,

With autosizing on I configured the changelog backend:

dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
cn: changelog
objectClass: top
objectClass: extensibleObject
objectClass: nsBackendInstance
nsslapd-suffix: cn=changelog
nsslapd-cachesize: -1
nsslapd-cachememsize: 8858370048
nsslapd-readonly: off
nsslapd-require-index: off
nsslapd-directory: /var/lib/dirsrv/slapd-CNVR-NET/db/changelog
nsslapd-dncachememsize: 3000000000

Here’s the ldif:

dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-dncachememsize
nsslapd-dncachememsize: 3000000000
-
replace: nsslapd-cachememsize
nsslapd-cachememsize: 3000000000

The server refused to change nsslapd-cachememsize because of autosizing but
nsslapd-dncachememsize did get changed. So it seems that I can still control
nsslapd-dncachememsize? When I restart 389-ds, the 3G persists as well.

Does that make sense?

Thank you for a quick response!

Sergei


>  On Mar 13, 2018, at 7:40 PM, William Brown
> <william(a)blackhats.net.au&gt; wrote:
>  
>  If autosize > 0, we write a new entrychace/cachememsize every start up.
>  
>  So you only need to set autosize to between 1 and 99 for it to work.
>  
>  There is some other logic in there to account for other scenarioes =
>  for example, if you set autosize to 0 AND you set the entry cachesize
>  to 0, we'll autosize it at start up anyway. BUT if you have autosize =
>  0, and entry cachesize > 0, we won't touch it.

[William Brown]

On Tue, 2018-03-13 at 21:10 -0500, Sergei Gerasenko wrote:
>  Hi William,
>  
>  With autosizing on I configured the changelog backend:
>  
>  dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
>  cn: changelog
>  objectClass: top
>  objectClass: extensibleObject
>  objectClass: nsBackendInstance
>  nsslapd-suffix: cn=changelog
>  nsslapd-cachesize: -1
>  nsslapd-cachememsize: 8858370048
>  nsslapd-readonly: off
>  nsslapd-require-index: off
>  nsslapd-directory: /var/lib/dirsrv/slapd-CNVR-NET/db/changelog
>  nsslapd-dncachememsize: 3000000000
>  
>  Here’s the ldif:
>  
>  dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
>  changetype: modify
>  replace: nsslapd-dncachememsize
>  nsslapd-dncachememsize: 3000000000
>  -
>  replace: nsslapd-cachememsize
>  nsslapd-cachememsize: 3000000000
>  
>  The server refused to change nsslapd-cachememsize because of
>  autosizing but nsslapd-dncachememsize did get changed. So it seems
>  that I can still control nsslapd-dncachememsize? When I restart 389-
>  ds, the 3G persists as well.
>  
>  Does that make sense? 
Yes it does.

>  
>  Thank you for a quick response! 
At the current point in time, dncachememsize is NOT controlled by
autosizing. There is some ideas in my head about how to improve this,
but the issue is legacy. We have one control: autosize percentage. But
this doesn't really represent a complete picture and story about how we
size our backends and data.

We have to continue to support the current variables, so I can't easily
change this from it's current form. 

*IF* I was given unlimited power (I never will be ;) ) I would probably
make the interface something like:



backend-foo:
autosize-max-memory: <value in bytes/mb/gb>
entrycache-slice: A%
dncache-slice: B%
dbcache-slice: C%
ndncache-slice: D%

So how would this work? 

You have autosize max that is the "total ram" we want to use. Lets say
2GB. Each backend would be given it's own "limit". We'd try to detect
some sane values on your system of course, but we can only do so much
:) So say ... each backend gets 5-10% of system ram limit. (We have to
be super conservative due to dogtag in freeipa which eats ram).

Then we slice that up. So entrycache-slice + dncache-slice + dbcache-
slice + ndncache-slice = 100%. So by default we might do something
like:

entrycache-slice: 75
dbcache-slice: 10
dncache-slice: 10
ndncache-slice: 5

So this on a 1GB backend translates to:

750MB of entrycache
100MB of dncache
100MB of dbcache
50MB of ndncache

of course some tests would be needed to find the right slice values by
default.


Then it's as simple as "if you want more from your backend" just up the
one memlimit value and "everything" increases in a correct proportional
rate. Rather than thinking about how to hand tune everything, we make
informed design decisions, you just say "here's mem max". You have
control to say each backend gets a different value (say FreeIPA you may
want 4GB to userRoot, 1GB to CA system backend). you still have lots of
flexibility as an admin, but you don't have to worry about so many
moving interacting parts. 


Hope that helps,

>  
>  Sergei
>  
>  
>  > On Mar 13, 2018, at 7:40 PM, William Brown <william(a)blackhats.net.a
>  > u> wrote:
>  > 
>  > If autosize > 0, we write a new entrychace/cachememsize every start
>  > up.
>  > 
>  > So you only need to set autosize to between 1 and 99 for it to
>  > work.
>  > 
>  > There is some other logic in there to account for other scenarioes
>  > =
>  > for example, if you set autosize to 0 AND you set the entry
>  > cachesize
>  > to 0, we'll autosize it at start up anyway. BUT if you have
>  > autosize =
>  > 0, and entry cachesize > 0, we won't touch it.
>  
>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
>  rg 
-- 
Thanks,

William Brown

[Sergei Gerasenko]

Thank you for the detailed response, William. That’s great info. You mentioned FreeIPA in
passing and that’s actually what I use 389-ds for. You mentioned dogtag eating memory. You
mean it has a memory leak or some other memory mismanagement issue? 

I have 125G of RAM on my systems. So, I can allocate quite a bit of RAM to the caches. My
plan is to set nsslapd-cache-autosize to 20% and set the dncache size to 5G for all three
backends. Does that sound reasonable to you? These are dedicated FreeIPA machines, so it’s
the main consumer of the memory.

Also, can I set the dncachesize once at the cn=config,cn=ldbm
database,cn=plugins,cn=config level and have all the other backends inherit that? Would I
have to remove that parameter from the other backends for it to become inherited?

Thanks again, William!
  Sergei
 
>  On Mar 18, 2018, at 8:06 PM, William Brown
> <william(a)blackhats.net.au&gt; wrote:
>  
>  On Tue, 2018-03-13 at 21:10 -0500, Sergei Gerasenko wrote:
> > Hi William,
> > 
> > With autosizing on I configured the changelog backend:
> > 
> > dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
> > cn: changelog
> > objectClass: top
> > objectClass: extensibleObject
> > objectClass: nsBackendInstance
> > nsslapd-suffix: cn=changelog
> > nsslapd-cachesize: -1
> > nsslapd-cachememsize: 8858370048
> > nsslapd-readonly: off
> > nsslapd-require-index: off
> > nsslapd-directory: /var/lib/dirsrv/slapd-CNVR-NET/db/changelog
> > nsslapd-dncachememsize: 3000000000
> > 
> > Here’s the ldif:
> > 
> > dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
> > changetype: modify
> > replace: nsslapd-dncachememsize
> > nsslapd-dncachememsize: 3000000000
> > -
> > replace: nsslapd-cachememsize
> > nsslapd-cachememsize: 3000000000
> > 
> > The server refused to change nsslapd-cachememsize because of
> > autosizing but nsslapd-dncachememsize did get changed. So it seems
> > that I can still control nsslapd-dncachememsize? When I restart 389-
> > ds, the 3G persists as well.
> > 
> > Does that make sense?
>  
>  Yes it does.
>  
> > 
> > Thank you for a quick response!
>  
>  At the current point in time, dncachememsize is NOT controlled by
>  autosizing. There is some ideas in my head about how to improve this,
>  but the issue is legacy. We have one control: autosize percentage. But
>  this doesn't really represent a complete picture and story about how we
>  size our backends and data.
>  
>  We have to continue to support the current variables, so I can't easily
>  change this from it's current form. 
>  
>  *IF* I was given unlimited power (I never will be ;) ) I would probably
>  make the interface something like:
>  
>  
>  
>  backend-foo:
>  autosize-max-memory: <value in bytes/mb/gb>
>  entrycache-slice: A%
>  dncache-slice: B%
>  dbcache-slice: C%
>  ndncache-slice: D%
>  
>  So how would this work? 
>  
>  You have autosize max that is the "total ram" we want to use. Lets say
>  2GB. Each backend would be given it's own "limit". We'd try to detect
>  some sane values on your system of course, but we can only do so much
>  :) So say ... each backend gets 5-10% of system ram limit. (We have to
>  be super conservative due to dogtag in freeipa which eats ram).
>  
>  Then we slice that up. So entrycache-slice + dncache-slice + dbcache-
>  slice + ndncache-slice = 100%. So by default we might do something
>  like:
>  
>  entrycache-slice: 75
>  dbcache-slice: 10
>  dncache-slice: 10
>  ndncache-slice: 5
>  
>  So this on a 1GB backend translates to:
>  
>  750MB of entrycache
>  100MB of dncache
>  100MB of dbcache
>  50MB of ndncache
>  
>  of course some tests would be needed to find the right slice values by
>  default.
>  
>  
>  Then it's as simple as "if you want more from your backend" just up the
>  one memlimit value and "everything" increases in a correct proportional
>  rate. Rather than thinking about how to hand tune everything, we make
>  informed design decisions, you just say "here's mem max". You have
>  control to say each backend gets a different value (say FreeIPA you may
>  want 4GB to userRoot, 1GB to CA system backend). you still have lots of
>  flexibility as an admin, but you don't have to worry about so many
>  moving interacting parts. 
>  
>  
>  Hope that helps,
>  
> > 
> > Sergei
> > 
> > 
> >> On Mar 13, 2018, at 7:40 PM, William Brown <william(a)blackhats.net.a
> >> u> wrote:
> >> 
> >> If autosize > 0, we write a new entrychace/cachememsize every start
> >> up.
> >> 
> >> So you only need to set autosize to between 1 and 99 for it to
> >> work.
> >> 
> >> There is some other logic in there to account for other scenarioes
> >> =
> >> for example, if you set autosize to 0 AND you set the entry
> >> cachesize
> >> to 0, we'll autosize it at start up anyway. BUT if you have
> >> autosize =
> >> 0, and entry cachesize > 0, we won't touch it.
> > 
> > _______________________________________________
> > 389-users mailing list -- 389-users(a)lists.fedoraproject.org
> > To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
> > rg
>  -- 
>  Thanks,
>  
>  William Brown
>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org

[William Brown]

On Sun, 2018-03-18 at 21:57 -0500, Sergei Gerasenko wrote:
>  Thank you for the detailed response, William. That’s great info. You
>  mentioned FreeIPA in passing and that’s actually what I use 389-ds
>  for. You mentioned dogtag eating memory. You mean it has a memory
>  leak or some other memory mismanagement issue?  
Dogtag is Java/Tomcat. It's well known for consuming large volumes of
ram!

I actually wrote the autotuning system specifically for FreeIPA since
it historically did no DS tuning. So I'm glad that you are finding it
useful! 

>  
>  I have 125G of RAM on my systems. So, I can allocate quite a bit of
>  RAM to the caches. My plan is to set nsslapd-cache-autosize to 20%
>  and set the dncache size to 5G for all three backends. Does that
>  sound reasonable to you? These are dedicated FreeIPA machines, so
>  it’s the main consumer of the memory. 
Sure, sounds reasonable to me - I'd want to see your database sizes to
make a complete assesment, but it seems pretty reasonable to me. 

>  
>  Also, can I set the dncachesize once at the cn=config,cn=ldbm
>  database,cn=plugins,cn=config level and have all the other backends
>  inherit that? Would I have to remove that parameter from the other
>  backends for it to become inherited? 
Sadly there is no method to inherit cache sizes.

Additionally, check that cn=config,cn=ldbm ... is actually
d'B'cachesize not d'N'cachesize.

Hope that helps! 

>  
>  Thanks again, William!
>    Sergei
>   
>  > On Mar 18, 2018, at 8:06 PM, William Brown <william(a)blackhats.net.a
>  > u> wrote:
>  > 
>  > On Tue, 2018-03-13 at 21:10 -0500, Sergei Gerasenko wrote:
>  > > Hi William,
>  > > 
>  > > With autosizing on I configured the changelog backend:
>  > > 
>  > > dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
>  > > cn: changelog
>  > > objectClass: top
>  > > objectClass: extensibleObject
>  > > objectClass: nsBackendInstance
>  > > nsslapd-suffix: cn=changelog
>  > > nsslapd-cachesize: -1
>  > > nsslapd-cachememsize: 8858370048
>  > > nsslapd-readonly: off
>  > > nsslapd-require-index: off
>  > > nsslapd-directory: /var/lib/dirsrv/slapd-CNVR-NET/db/changelog
>  > > nsslapd-dncachememsize: 3000000000
>  > > 
>  > > Here’s the ldif:
>  > > 
>  > > dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config
>  > > changetype: modify
>  > > replace: nsslapd-dncachememsize
>  > > nsslapd-dncachememsize: 3000000000
>  > > -
>  > > replace: nsslapd-cachememsize
>  > > nsslapd-cachememsize: 3000000000
>  > > 
>  > > The server refused to change nsslapd-cachememsize because of
>  > > autosizing but nsslapd-dncachememsize did get changed. So it
>  > > seems
>  > > that I can still control nsslapd-dncachememsize? When I restart
>  > > 389-
>  > > ds, the 3G persists as well.
>  > > 
>  > > Does that make sense?
>  > 
>  > Yes it does.
>  > 
>  > > 
>  > > Thank you for a quick response!
>  > 
>  > At the current point in time, dncachememsize is NOT controlled by
>  > autosizing. There is some ideas in my head about how to improve
>  > this,
>  > but the issue is legacy. We have one control: autosize percentage.
>  > But
>  > this doesn't really represent a complete picture and story about
>  > how we
>  > size our backends and data.
>  > 
>  > We have to continue to support the current variables, so I can't
>  > easily
>  > change this from it's current form. 
>  > 
>  > *IF* I was given unlimited power (I never will be ;) ) I would
>  > probably
>  > make the interface something like:
>  > 
>  > 
>  > 
>  > backend-foo:
>  > autosize-max-memory: <value in bytes/mb/gb>
>  > entrycache-slice: A%
>  > dncache-slice: B%
>  > dbcache-slice: C%
>  > ndncache-slice: D%
>  > 
>  > So how would this work? 
>  > 
>  > You have autosize max that is the "total ram" we want to use. Lets
>  > say
>  > 2GB. Each backend would be given it's own "limit". We'd try to
>  > detect
>  > some sane values on your system of course, but we can only do so
>  > much
>  > :) So say ... each backend gets 5-10% of system ram limit. (We have
>  > to
>  > be super conservative due to dogtag in freeipa which eats ram).
>  > 
>  > Then we slice that up. So entrycache-slice + dncache-slice +
>  > dbcache-
>  > slice + ndncache-slice = 100%. So by default we might do something
>  > like:
>  > 
>  > entrycache-slice: 75
>  > dbcache-slice: 10
>  > dncache-slice: 10
>  > ndncache-slice: 5
>  > 
>  > So this on a 1GB backend translates to:
>  > 
>  > 750MB of entrycache
>  > 100MB of dncache
>  > 100MB of dbcache
>  > 50MB of ndncache
>  > 
>  > of course some tests would be needed to find the right slice values
>  > by
>  > default.
>  > 
>  > 
>  > Then it's as simple as "if you want more from your backend" just up
>  > the
>  > one memlimit value and "everything" increases in a correct
>  > proportional
>  > rate. Rather than thinking about how to hand tune everything, we
>  > make
>  > informed design decisions, you just say "here's mem max". You have
>  > control to say each backend gets a different value (say FreeIPA you
>  > may
>  > want 4GB to userRoot, 1GB to CA system backend). you still have
>  > lots of
>  > flexibility as an admin, but you don't have to worry about so many
>  > moving interacting parts. 
>  > 
>  > 
>  > Hope that helps,
>  > 
>  > > 
>  > > Sergei
>  > > 
>  > > 
>  > > > On Mar 13, 2018, at 7:40 PM, William Brown <william(a)blackhats.n
>  > > > et.a
>  > > > u> wrote:
>  > > > 
>  > > > If autosize > 0, we write a new entrychace/cachememsize every
>  > > > start
>  > > > up.
>  > > > 
>  > > > So you only need to set autosize to between 1 and 99 for it to
>  > > > work.
>  > > > 
>  > > > There is some other logic in there to account for other
>  > > > scenarioes
>  > > > =
>  > > > for example, if you set autosize to 0 AND you set the entry
>  > > > cachesize
>  > > > to 0, we'll autosize it at start up anyway. BUT if you have
>  > > > autosize =
>  > > > 0, and entry cachesize > 0, we won't touch it.
>  > > 
>  > > _______________________________________________
>  > > 389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  > > To unsubscribe send an email to 389-users-leave(a)lists.fedoraproje
>  > > ct.o
>  > > rg
>  > 
>  > -- 
>  > Thanks,
>  > 
>  > William Brown
>  > _______________________________________________
>  > 389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  > To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject
>  > .org
>  
>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
>  rg 
-- 
Thanks,

William Brown

[Sergei Gerasenko]

>  Dogtag is Java/Tomcat. It's well known for consuming large
> volumes of
>  ram! 
Ah, I thought it was something besides that :)

>  Sure, sounds reasonable to me - I'd want to see your database
> sizes to
>  make a complete assesment, but it seems pretty reasonable to me.  
I will get that for you. I’ve used dbmon.sh on a live system and things looked pretty well
(>=92%). You would like to see the number of entries in the db, right?

>  Additionally, check that cn=config,cn=ldbm ... is actually
>  d'B'cachesize not d’N'cachesize. 
Now I’m confused a bit. I thought the dncachesize was what was supposed to be set manually
while everything else was autosized. Sorry for being slow on this.

[William Brown]

On Mon, 2018-03-19 at 20:58 -0500, Sergei Gerasenko wrote:
>  > Dogtag is Java/Tomcat. It's well known for consuming large
> volumes
>  > of
>  > ram!
>  
>  Ah, I thought it was something besides that :) 
Nope, just that :) 

>  
>  > Sure, sounds reasonable to me - I'd want to see your database sizes
>  > to
>  > make a complete assesment, but it seems pretty reasonable to me. 
>  
>  I will get that for you. I’ve used dbmon.sh on a live system and
>  things looked pretty well (>=92%). You would like to see the number
>  of entries in the db, right?
>   
Yeah I think that should do it. It would be good to know the filesizes
in the backend too. 

>  > Additionally, check that cn=config,cn=ldbm ... is actually
>  > d'B'cachesize not d’N'cachesize.
>  
>  Now I’m confused a bit. I thought the dncachesize was what was
>  supposed to be set manually while everything else was autosized.
>  Sorry for being slow on this. 
yeah, dncachesize is manual. But I think dncachesize is per backend,
not part of cn=config,cn=ldbm plugin. 

>  _______________________________________________
>  389-users mailing list -- 389-users(a)lists.fedoraproject.org
>  To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
>  rg 
-- 
Thanks,

William Brown

[Sergei Gerasenko]

Hi William,

>  On Mar 19, 2018, at 9:18 PM, William Brown
> <william(a)blackhats.net.au&gt; wrote:
>  
>  yeah, dncachesize is manual. But I think dncachesize is per backend,
>  not part of cn=config,cn=ldbm plugin.  
Yes, I see one for changelog and one for userRoot.

Here’s the data:

>  dbscan -f /var/lib/dirsrv/slapd-XXXX-XXX/db/userRoot/id2entry.db -t
> 200 | grep -c rdn: 
12212

>  dbmon.sh 
Tue Mar 20 02:16:04 UTC 2018
dbcachefree 432873472 free% 80.629 roevicts 0 hit% 99 pagein 1267 pageout 23191
changelog:ent         61    6509293897  100.0  4359.2
changelog:dn        2926    2999793332  100.0    70.6
 userroot:ent      11777    6408341275   98.4  8594.6
 userroot:dn       11777    2998722450  100.0   108.5

>  ls -lh /var/lib/dirsrv/slapd-XXXX-XXX/db/userRoot 
total 121M
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:41 aci.db
-rw------- 1 dirsrv dirsrv 744K Mar 19 21:04 ancestorid.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:41 automountkey.db
-rw------- 1 dirsrv dirsrv 5.7M Mar 19 18:39 cn.db
-rw------- 1 dirsrv dirsrv   51 Mar 19 14:44 DBVERSION
-rw------- 1 dirsrv dirsrv 296K Mar 19 14:41 displayname.db
-rw------- 1 dirsrv dirsrv 4.7M Mar 19 21:04 entryrdn.db
-rw------- 1 dirsrv dirsrv 144K Mar 20 02:23 entryusn.db
-rw------- 1 dirsrv dirsrv 592K Mar 19 18:15 fqdn.db
-rw------- 1 dirsrv dirsrv  56K Mar 19 14:41 gidnumber.db
-rw------- 1 dirsrv dirsrv 176K Mar 19 14:41 givenName.db
-rw------- 1 dirsrv dirsrv  74M Mar 20 02:23 id2entry.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipaallowedtarget.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipaassignedidview.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 18:15 ipakrbprincipalalias.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipalocation.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipaMemberCa.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:41 ipaMemberCertProfile.db
-rw------- 1 dirsrv dirsrv 112K Mar 19 14:41 ipasudorunas.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipasudorunasgroup.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 ipatokenradiusconfiglink.db
-rw------- 1 dirsrv dirsrv 952K Mar 19 21:04 ipauniqueid.db
-rw------- 1 dirsrv dirsrv 2.8M Mar 19 18:15 krbCanonicalName.db
-rw------- 1 dirsrv dirsrv 7.2M Mar 19 18:15 krbPrincipalName.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 macAddress.db
-rw------- 1 dirsrv dirsrv 416K Mar 19 14:41 mail.db
-rw------- 1 dirsrv dirsrv 8.1M Mar 19 18:15 managedby.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 manager.db
-rw------- 1 dirsrv dirsrv 264K Mar 19 21:04 memberallowcmd.db
-rw------- 1 dirsrv dirsrv 3.3M Mar 19 19:09 member.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 memberdenycmd.db
-rw------- 1 dirsrv dirsrv 1.7M Mar 19 14:41 memberHost.db
-rw------- 1 dirsrv dirsrv 2.0M Mar 19 19:09 memberOf.db
-rw------- 1 dirsrv dirsrv 128K Mar 19 14:41 memberservice.db
-rw------- 1 dirsrv dirsrv 472K Mar 19 14:41 memberUser.db
-rw------- 1 dirsrv dirsrv  48K Mar 19 20:12 nscpEntryDN.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:41 nsds5ReplConflict.db
-rw------- 1 dirsrv dirsrv  32K Mar 19 20:12 nsTombstoneCSN.db
-rw------- 1 dirsrv dirsrv 920K Mar 19 21:04 nsuniqueid.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 21:04 numsubordinates.db
-rw------- 1 dirsrv dirsrv 1.3M Mar 19 21:04 objectclass.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:41 ou.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 owner.db
-rw------- 1 dirsrv dirsrv 176K Mar 19 21:04 parentid.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 secretary.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 seeAlso.db
-rw------- 1 dirsrv dirsrv 200K Mar 19 14:41 sn.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 sourcehost.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 18:39 telephoneNumber.db
-rw------- 1 dirsrv dirsrv  96K Mar 19 14:41 title.db
-rw------- 1 dirsrv dirsrv 184K Mar 19 14:41 uid.db
-rw------- 1 dirsrv dirsrv  56K Mar 19 14:41 uidnumber.db
-rw------- 1 dirsrv dirsrv  16K Mar 19 14:44 uniquemember.db
-rw------- 1 dirsrv dirsrv 4.4M Mar 19 18:15 userCertificate.db

Thank you!
  Sergei

[William Brown]

>  
>  > dbmon.sh
>  Tue Mar 20 02:16:04 UTC 2018
>  dbcachefree 432873472 free% 80.629 roevicts 0 hit% 99 pagein 1267
>  pageout 23191
>  changelog:ent         61    6509293897  100.0  4359.2
>  changelog:dn        2926    2999793332  100.0    70.6
>   userroot:ent      11777    6408341275   98.4  8594.6
>   userroot:dn       11777    2998722450  100.0   108.5
>  
>  > ls -lh /var/lib/dirsrv/slapd-XXXX-XXX/db/userRoot
>  total 121M 
Yep with a few GB of cache you'll be able to double or triple your
number of entries easily. I think you will have few issues from here.

Thanks! 

> Thank you!
>   Sergei
> _______________________________________________
> 389-users mailing list -- 389-users(a)lists.fedoraproject.org
> To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.o
> rg