On Wed, Aug 31, 2022 at 3:57 PM Stephen Smoogen ssmoogen@redhat.com wrote:
On Tue, 30 Aug 2022 at 14:16, Mikolaj Izdebski mizdebsk@redhat.com wrote:
On Thu, Aug 25, 2022 at 9:53 AM Miro HronĨok mhroncok@redhat.com wrote:
To submit more scratch builds we would need larger builder capacity. This doesn't necessarily mean more or better hardware. Better Koji configuration would help a lot. We have some very powerful builders with up to 224 processors, but their capacity is set to 2. This means that the builder stops accepting new tasks once load gets to 2, which is less than 1 %.
Example buildhw-a64-20.iad2.fedoraproject.org Capacity is 2, check with: koji hostinfo buildhw-a64-20.iad2.fedoraproject.org 224 CPUs, load average: 2.04, 2.07, 2.05 memory: 251Gi total, 7.3Gi used, 242Gi available Yet, at the time of writing the builder is marked as not ready (!!) for taking more builds due to exceeded capacity.
This is not an individual case, we have many builders like that.
I believe there are 2 builders with 224 CPU's. They are both 'prototype' aarch64 systems we got on loan from a vendor and are tempermental. They do not have good disk IO and do not have the capability of increasing the disk IO. This means that you have a very very fast cpu and slow as treacle disk io. The network on them has also been 'fun' where if the net access got too high, it would require a complete power cycle with the plugs pulled out because the BMC is not licensed.
Slow disk could be worked around by using tmpfs for mock chroots - there is lots of free memory. I was not aware of the networking issues.
-- Mikolaj Izdebski
Most of the 'big' builders I remember are similar in hardware.. They look great on paper, but if you try to give them a large capacity you end up with even slower builds because they are loaners or spares which are good for specific workloads and not general usage.
-- Stephen Smoogen, Red Hat Automotive Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue