Freeze break request: decrease mm crawler threads

[Patrick Uiterwijk]

Hi all,

Can I get retroactive +1s for the following patch?
Given the imminent Fedora 23 release, we want up-to-date mirrorlists,
so I will push this patch ahead of +1s to prevent having to depend on
people in the weekend.

This tunes the number of threads down for the mm crawler to 23.
This is probably lower than we can handle, but I'm setting it this low to
make sure that the crawls finish for the F23 release.
After release we should re-evaluate the number of threads used.



commit d23d2496522923d25e1ffb04d82a0b986387f236
Author: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;
Date:   Sun Nov 1 01:04:35 2015 +0000

    Decrease the number of mm_crawler threads
    
    It seems like an update caused the crawler to use slightly
    more memory than before, meaning the previous tuning of
    27 threads no longer fits in the server's memory.
    This patch brings it down to 23, which is for now known-good.
    We should look again at what values to use after freeze.
    
    Signed-off-by: Patrick Uiterwijk <puiterwijk(a)redhat.com&gt;

diff --git a/roles/mirrormanager/crawler/files/crawler.cron
b/roles/mirrormanager/crawler/files/crawler.cron
index 8ace23d..a3691d7 100644
--- a/roles/mirrormanager/crawler/files/crawler.cron
+++ b/roles/mirrormanager/crawler/files/crawler.cron
@@ -5,4 +5,4 @@
 # [ "`hostname -s`" == "mm-crawler02" ] && sleep 2h is used
to start the crawl
 # later on the second crawler to reduce the number of parallel accesses to
 # the database
-0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ]
&& sleep 2h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 27
`/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
+0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ]
&& sleep 2h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 23
`/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1



With kind regards,
Patrick Uiterwijk
Fedora Infra