Dan Horák píše v Čt 22. 07. 2010 v 11:44 +0200:
Hi all,
there are some changes that needs to be done on the Fedora/s390x
infrastructure for various reasons and we should prepare a plan how and
when to do them.
Tasks
-----
1. upgrade builders to something with kernel 2.6.32+ like EL-6 beta
Yesterday a new glibc was built in primary Fedora, it drops some
compatibility stuff for old kernels and now requires kernel 2.6.32+ on
the builders.
done, due some changes python's urllib2 (or something else) I had to
apply a workaround for kojid when it reads rpm header from network
2. redesign resource allocation for the builders
The builders (and the squid cache) don't behave very well when under
full load. Timeouts when downloading the buildroots thru the cache are
quite rare (but they happen every day). Other observed behaviour looks
like a completely swapped out guest, it doesn't respond to ping, koji
doesn't update its status on the hub and it can take minutes, maybe even
tens of minutes, before the machine starts responding again.
Unfortunately often this behaviour means stuck builds and manually
restarting the builder daemon.
I think part of this issue could be solved on the z/VM side (size of
RAM, number of CPUs per builder), part could be tuning the koji
configuration (max jobs per builder, max load, parallel make, ...). Also
interesting would be to see performance/resource usage statistics from
z/VM.
switched to use only 3 builders, let's wait how it will behave
3. rebuild the storage on the hub
The sub-optimal storage configuration is known for some time, but there
is still room on the disks for few months of work (my guess).
4. upgrade Koji to 1.4
Here we should be on par with the primary Fedora buildsystem.
done
5. update builder configs for dist-git
done
With these steps done we are on par with the primary Fedora and can
continue building new stuff.
But a new issue have appeared when I started koji-shadow, could be
related to the koji upgrade ...
Dan