F20 System Wide Change: Perl 5.18

[Paul Howarth]

On 08/08/13 12:50, Petr Pisar wrote:
> On 2013-08-08, Paul Howarth <paul at city-fan.org> wrote:
>> On 08/08/13 10:01, Phil Knirsch wrote:
>>>
>>> Just like Seth's script it has a few obviously problems as for really
>>> correct build ordering you need to flip-flop between srpms/specfiles and
>>> binary rpms, but that will need a bit of a different approach. As it
>>> stands right now some of the dependencies it can't detect and resolve
>>> are build time generated ones, which might a showstopper for some uses
>>> of it.
>>>
>>> I've tested the version i have here with several complete past Fedora
>>> trees, for texlive and KDE rebuilds and updates and for all the cases
>>> the output look pretty sane.
>>>
>>> Hope this helps,
>>
>> We already know where it's beneficial to break build dependency cycles
>> and have built that into the spec files, triggered by the
>> %{perl_bootstrap} macro being set (which it is in the current Rawhide
>> perl). So a build ordering script for perl bootstrapping should take
>> that into account.
>>
> And therefore my slow script considers SRPM as well as binary packages
> distinctively. Therefore it clones git repostories, creates SRPMs in mock
> environment (because data from koji will differ from to-rebuild packages
> due to the perl_bootstrap macro), retrieves just built binary packages,
> extracts the requires/provides and solves them on its own. Of course it
> submits whole groups of packages to rebuild and it waits for build root
> rotation. It does prety much everything what is needed.

My own script does something similar but doesn't build the packages, 
since there are hardly any for which the runtime dependencies of 
bootstrapped packages are different from those of non-bootstrapped 
packages - I have a separate override configuration file in which I 
specify those differences. Needless to say, this saves a lot of time. So 
the git checkout is used to generate SRPMs, and I get the buildreqs from 
those, and using the existing Rawhide metadata for the runtime dependencies.

> (Curriously, this year I could not build in 16 threads as last year,
> becuse dist-git starts refusing on 10 threads. Also after ausil started
> his mass rebuild, my koji client started getting host unreachable errors.)
>
> However the really slow part is the dependency solver. I had no time to
> improve it before this rebuild. I will do before next time. But now I'm
> not going to change it becuse it's to complicated.

I did my depsolver in python, leveraging code from yum. It's actually 
quite quick, much quicker than extracting the dependency data from git, 
except in the presence of circular build dependencies, but it does a 
decent job of identifying and prioritizing those, and I keep on top of 
them by running the script periodically and raising bugs where needed.

> Neverthless we are looming to the point where no new packages will be
> possible to rebuild due to unsatisified depenencies. We are now at
> 95 % done. The rest will need fixing or dropping
> <https://fedoraproject.org/wiki/Changes/perl5.18#Items_to_be_done>. Once
> we get there, I will publish list of failed and unfinished packages
> together with dependency graph.
>
> I have already done 169 changes in the F20 packages to achieve those
> 95 % and I'm quite tired and sick of all of that. (Mainly
> because maintainers do not upgrade their packages or because they do not
> declare all needed dependencies). (I'm thinking of running dummy
> rebuilds during a year regularly, to find all these buggy packages before
> next Perl rebuild.)

It's a good plan. There are people running through all modules on CPAN 
and raising bugs where they break with bleadperl but if upstreams don't 
fix these bugs there's not much we can do.

Paul.