Dear Conductor developers, Tired of running endless rspec and cucumber tests and having to wait almost forever for them to finnish? Here is (maybe) the solution! :)
As you know, RSpec tests are very database heavy, so it probably does make sense to put postgresql data directory into a ramdisk-based filesystem. The same applies to cucumber tests, but there is a difference that cucumber is much more solr-heavy than db-heavy. Let's then put a solr data directory into a tmpfs and see what happens.
Here are some "benchmarks" for you to have a picture of what's going on.
RSpec: With pgsql on tmpfs: $ rake Finished in 79.749916 seconds
With pgsql on ext4 (and luks): $ rake Finished in 470.17935 seconds
So, rspec is on tmpfs about 489.5 percent faster.
Cucumber: With solr/data on tmpfs and pgsql on ext4 $ rake cucumber 1m47.025s
With solr/data on tmpfs and pgsql on tmpfs $ rake cucumber 1m44.647s
This is not a significant improvement compared to last result.
With solr/data on ext4 and pgsql on tmpfs $ rake cucumber 5m18.378s
Well, it seems that cucumber is really solr-heavy.
And for completness, here are tests with both solr and pgsql on ext4, which is probably the most common setup: $ rake cucumber 5m24.583s
Well, it looks like the solr on tmpfs can provide ~156.5 percent speed improvement. There are also scripts for automated setup. Please note that these scripts should be run from cucumber/src directory. Please, give it a shot and also any feedback would be really appreciated.
Best regards, Maros Zatko mzatko@redhat.com
#!/bin/bash -x
if ! mount | grep /var/lib/pgsql; then /sbin/service postgresql stop
mount -t tmpfs tmpfs /var/lib/pgsql
chown postgres:postgres /var/lib/pgsql chmod 700 /var/lib/pgsql
/sbin/service postgresql initdb
cat > /var/lib/pgsql/data/pg_hba.conf <<EOL local all all trust host all all 127.0.0.1/32 trust host all all ::1/128 ident EOL
/sbin/service postgresql start su - postgres -c "echo -e "CREATE USER dcloud WITH PASSWORD 'v23zj59an' CREATEDB;\n\q" | psql"
rake dc:setup
else echo Postgresql is already on tmpfs fi
#!/bin/bash -x
if ! mount | grep solr/data ; then
rake sunspot:solr:stop RAILS_ENV=cucumber rake sunspot:solr:stop
sudo mount -t tmpfs tmpfs solr/data
rake sunspot:solr:start rake sunspot:solr:start RAILS_ENV=cucumber rake sunspot:reindex
else echo Solr/data is already on tmpfs fi
#!/bin/bash -x
if mount | grep /var/lib/pgsql; then /sbin/service postgresql stop umount /var/lib/pgsql /sbin/service postgresql start else echo Postgresql is not on tmpfs fi
#!/bin/bash -x
if mount | grep solr/data ; then
rake sunspot:solr:stop RAILS_ENV=cucumber rake sunspot:solr:stop
sudo umount solr/data
rake sunspot:solr:start rake sunspot:solr:start RAILS_ENV=cucumber rake sunspot:reindex
else echo Solr/data is not on tmpfs fi
aeolus-devel@lists.fedorahosted.org