On 11/10/2017 07:01 AM, Richard W.M. Jones wrote:
On Fri, Nov 10, 2017 at 06:33:09AM -0800, John Reiser wrote:
> On 11/10/2017 1058Z, Richard W.M. Jones wrote:
>> 24 hours and counting ...
>> I talked to someone on #fedora-admin about this and it is caused by
>> Koji having to load the metadata (files, dependencies etc) of every
>> RPM into its database. Combined with the fact this happens to be
>> running on s390x and it's texlive, makes it slow.
> Can you be more specific? Is it slow because:
> 1. the s390x has a slow or inappropriate CPU for the task?
> 2. the VM on the s390x has not enough real RAM, and is page thrashing on a spinning
> 3. the database uses a quadratic algorithm that hurts in this case?
> 4. the database has very high CPU overhead in general?
> 5. "every RPM" means all 30,000 .rpm in Fedora, not just the 300 for
> (And if so, then why isn't tagging slow for _every_ package?)
> 6. some other specific reason(s)?
> Indexing 1GB of data should take no more than a few minutes.
I don't have any access to the machine so I'm not in a position to say.
I will note however that the previous successful build of texlive took
4 hours in total (that's building, adding to the database and
tagging). This one has been running for 30 hours and still has not
The S390x builders are not in the same datacenter as all the rest of the
builders, the koji hub and it's database. They are not particularly
slow, but sending all the data back and forth over the net is likely the
It's particularly bad (if I understand correctly how it works) because
one builder manages the parent task and farms out the builds to other
builders, then collects them back and uploads them to the hub. So, if a
s390x builder is controlling the parent task it will have to pull all
the resuts back accross the net to itself, then push them back accross
the net to the hub.
All that said, I don't see where s390x is involved in the current build.
The parent task is being handled by buildvm-23, but indeed it seems like
something has gone wrong. I can try and free that task to another
builder, but that might cause it to fail. I suppose I can resubmit after
that... so will give it a try