Speed hacks for testers

[Simo Sorce]

I hadn't run make check in a while while and today I discovered it was
horrendously slow. The problem lies almost entirely in the fact that we
use fsync a lot in all tests, this is because we want a "safe" behavior
for our DB, so that a system crash will not also crash the db
consistency and, at most, loose the current transaction.

This is all and well in production (and we may later want to discuss
which databases we really want to preserve at all costs), but to
perform functional test it sucks badly.

Attached find a patch that I am NOT proposing for inclusion, but that
will help all poor souls that need to do just tests.
If someone have a good idea on how to do that same thing but only
conditionally for tests (or maybe through a sssd.conf option) please
let me know.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

[Stephen Gallagher]

Simo, see my recent patch to add a --with-test-dir. If you set this
to /dev/shm, it increases performance immensely.


On Feb 27, 2010, at 4:36 PM, Simo Sorce <ssorce(a)redhat.com&gt; wrote:

>  I hadn't run make check in a while while and today I discovered it was
>  horrendously slow. The problem lies almost entirely in the fact that
>  we
>  use fsync a lot in all tests, this is because we want a "safe"
>  behavior
>  for our DB, so that a system crash will not also crash the db
>  consistency and, at most, loose the current transaction.
>
>  This is all and well in production (and we may later want to discuss
>  which databases we really want to preserve at all costs), but to
>  perform functional test it sucks badly.
>
>  Attached find a patch that I am NOT proposing for inclusion, but that
>  will help all poor souls that need to do just tests.
>  If someone have a good idea on how to do that same thing but only
>  conditionally for tests (or maybe through a sssd.conf option) please
>  let me know.
>
>  Simo.
>
>  --
>  Simo Sorce * Red Hat, Inc * New York
>  <0001-HACK-Speed-hacks.patch>
>  _______________________________________________
>  sssd-devel mailing list
>  sssd-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/sssd-devel

[Simo Sorce]

On Sat, 27 Feb 2010 17:37:27 -0500
Stephen Gallagher <stephen(a)gallagherhome.com&gt; wrote:

>  Simo, see my recent patch to add a --with-test-dir. If you set this
>  to /dev/shm, it increases performance immensely.
>   
Yes, I've seen that, but NOSYNC seem a better way then hacking up a RAM
disk ... we may even think of adding a propoer option, for those that
need the performance even if it may cost them consistency in case of a
hard failure. I am sure on servers this is a tradeoff you may want
because the sustained cost of NOSYNC would be probably higher than the
occasional crash+repopulate cache. Of course if NOSYNC is used we need
to be able to detect that the Cache DB is corrupted and wipe it at
startup.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/27/2010 09:17 PM, Simo Sorce wrote:
>  On Sat, 27 Feb 2010 17:37:27 -0500
>  Stephen Gallagher <stephen(a)gallagherhome.com&gt; wrote:
>  
> > Simo, see my recent patch to add a --with-test-dir. If you set this
> > to /dev/shm, it increases performance immensely.
> >
>  
>  Yes, I've seen that, but NOSYNC seem a better way then hacking up a RAM
>  disk ... we may even think of adding a propoer option, for those that
>  need the performance even if it may cost them consistency in case of a
>  hard failure. I am sure on servers this is a tradeoff you may want
>  because the sustained cost of NOSYNC would be probably higher than the
>  occasional crash+repopulate cache. Of course if NOSYNC is used we need
>  to be able to detect that the Cache DB is corrupted and wipe it at
>  startup.
>  
>  Simo.
>   
To answer your original question, we already have a define in SSSD to
tell when we're running as a test or not. sysdb_domain_init_internal()
has a section within #ifdef SYSDB_TEST that directs SSSD to use the
build dir for the memberof plugin, rather than the system dir. It would
be fairly easy to extend this to handle the ldb_connect call as well.

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkuM/GQACgkQeiVVYja6o6NWIACdEPRD6PpqXknO9rav1RExg8kY
> ULkAn34HeS8KNUUBozdTBETUIggpicON
> =eukp
> -----END PGP SIGNATURE-----