F20 System Wide Change: Perl 5.18

[Paul Howarth]

On 08/08/13 10:01, Phil Knirsch wrote:
> On 08/07/2013 10:56 AM, Marcela Mašláňová wrote:
>> On 08/07/2013 03:01 AM, Dennis Gilmore wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> On Tue, 06 Aug 2013 13:39:31 -0700
>>> Adam Williamson <awilliam at redhat.com> wrote:
>>>
>>>> On Thu, 2013-06-13 at 11:10 -0600, Kevin Fenzi wrote:
>>>>> On Thu, 13 Jun 2013 13:23:51 +0000 (UTC)
>>>>> Petr Pisar <ppisar at redhat.com> wrote:
>>>>>
>>>>>> On 2013-06-12, Kevin Fenzi <kevin at scrye.com> wrote:
>>>>>>> So, there's nothing preventing the side tag and rebuild anytime
>>>>>>> now right? 5.18.0 is out, so we could start that work in
>>>>>>> rawhide?=20
>>>>>>>
>>>>>> Currently 5.18.0 does not pass one test when running in mock and
>>>>>> koji. (It's because of the terminal usage in tested perl
>>>>>> debugger.) We think we could have solved this issue in a few days.
>>>>>
>>>>> Cool.
>>>>>
>>>>>> Could you explain how the side tag inheritance works? It inherits
>>>>>> everything from rawhide, even builds made after the side tag
>>>>>> creation,
>>>>>
>>>>> yes.
>>>>>
>>>>>> except packages whose builds have been already made in the side
>>>>>> tag. Am I right? That means we still get fresh third-party
>>>>>> dependencies from rawhide.
>>>>>
>>>>> yes. However, there's are several downsides:
>>>>>
>>>>> - Each side tag adds newrepo tasks which increases load a lot.
>>>>> - If you rebuild perl-foo-1.0-1 in the side tag against the new
>>>>> perl, then the maintainer has to fix something in rawhide, they
>>>>> would build perl-foo-1.0-2 in rawhide and when the side tag was
>>>>> merged back over either everyone would get the older one with the
>>>>> bug, or the newer one against the old perl. So, it's really
>>>>> important to not take a long time using a side tag to avoid this
>>>>> problem as much as possible.
>>>>
>>>> Seems like this one came true in practice. It seems like a 5.18
>>>> rebuild run was done in a side tag and then merged back into Rawhide.
>>>> Unfortunately, quite a lot of the 5.18 rebuilds seem to have been done
>>>> prior to the general F20 mass rebuild - so the mass rebuild won out,
>>>> and effectively squelched the perl rebuild.
>>>
>>> The f20-perl tag was merged back before the mass rebuild was started.
>>> so everything in the mass rebuild was built against the new perl.
>>> however because the perl rebuild was at a week there was quite a few
>>> packages rebuilt against the old perl. we need to work out how to build
>>> perl quicker. your analysis is not really correct.
>>>
>>> Dennis
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v2.0.19 (GNU/Linux)
>>>
>>> iEYEARECAAYFAlIBnHcACgkQkSxm47BaWfeWKgCeKSTmyV0Yrn9oulqe3UfCgEFH
>>> ZxgAn3vU40FXlBwcxg7hRpyx40OeGdVN
>>> =fMp/
>>> -----END PGP SIGNATURE-----
>>>
>> If someone knows about a tool, which can give build order faster than
>> Petr's tool, then it would help ;-)
>>
>> Marcela
>
> I've been working on improving one of Seth's last scripts call
> buildorder that he wrote to order a set or SRPMS[1].
>
> The current version is far from complete and/or bugfree yet, but i've
> attached the current version to this email.
>
> It's really simple: You give it a set of srpms and it spits out the
> buildorder for it as precisely as it can get it.
>
> The improvements i made was to get rid of the topology sort module he
> had been using and replaced it with my own implementation which also
> automatically breaks build loops at spots that are likely to be most
> beneficial to be broken. It does grouping as well, just like Seth's
> original version, but the actual group output is commented out at the
> moment as i'm using it for sequentially rebuilding things, not in
> parallel, so groups don't matter for my use case. If you want groups
> simply remove the comment from the
>
> # print "Group %d (%d reqs)" % (group, minreqs)
>
> line.
>
> It also fixes an issue with the global macro definitions which didn't
> get cleared.
>
> I've also added the manual provides specified in the specfiles, that
> gave a bit better dependency completion.
>
> Last (and least) there is some commented out code it in where you can
> potentially use this code to also find the full buildchain for a
> specific package, basically allowing you to specify 1 or a few srpms
> you'd like to build and the script would spit out a list of all srpms
> you'll need to build for those packages in the necessary order.
>
> Just like Seth's script it has a few obviously problems as for really
> correct build ordering you need to flip-flop between srpms/specfiles and
> binary rpms, but that will need a bit of a different approach. As it
> stands right now some of the dependencies it can't detect and resolve
> are build time generated ones, which might a showstopper for some uses
> of it.
>
> I've tested the version i have here with several complete past Fedora
> trees, for texlive and KDE rebuilds and updates and for all the cases
> the output look pretty sane.
>
> Hope this helps,

We already know where it's beneficial to break build dependency cycles 
and have built that into the spec files, triggered by the 
%{perl_bootstrap} macro being set (which it is in the current Rawhide 
perl). So a build ordering script for perl bootstrapping should take 
that into account.

Paul.