Using Beaker to test a mass rebuild of Fedora Python packages against Python 3.5

[Nick Coghlan]

With Python 3.5 in beta upstream, I'd like to create a Beaker recipe that:

1. Rebuilds the Fedora Python RPMs using the upstream Python 3.5 beta
tarball rather than the stable 3.4 release
2. Queries the Fedora repos to get the list of all packages with a build
or runtime requirement on Python 3
3. Uses
https://beaker-project.org/docs-develop/user-guide/beaker-provided-tasks....
to rebuild all the packages from 2 against the Python 3.5 beta RPM from 1

This idea pushes the limits of my current rpm/yum/dnf foo though, so I
figured I'd ask for advice here before I started hacking away at the
problem :)

Cheers,
Nick.

-- 
Nick Coghlan
Red Hat PnT Operations
DevOps Enablement, Brisbane

Software Development Workflow Designer & Process Architect

[Matej Stuchlik]

----- Original Message -----
>  From: "Nick Coghlan" <ncoghlan(a)redhat.com&gt;
>  To: "beaker-devel" <beaker-devel(a)lists.fedorahosted.org&gt;
>  Cc: "Matej Stuchlik" <mstuchli(a)redhat.com&gt;
>  Sent: Thursday, June 25, 2015 7:05:44 AM
>  Subject: Using Beaker to test a mass rebuild of Fedora Python packages against Python
> 3.5
>  
>  With Python 3.5 in beta upstream, I'd like to create a Beaker recipe that:
>  
>  1. Rebuilds the Fedora Python RPMs using the upstream Python 3.5 beta
>  tarball rather than the stable 3.4 release
>  2. Queries the Fedora repos to get the list of all packages with a build
>  or runtime requirement on Python 3 
I believe this part can be achieved with:
% dnf install fedora-repos-rawhide
% dnf repoquery --arch=src --disablerepo='*' --enablerepo=rawhide-source
--whatrequires python3-devel | wc -l          
783

>  3. Uses
>
> https://beaker-project.org/docs-develop/user-guide/beaker-provided-tasks....
>  to rebuild all the packages from 2 against the Python 3.5 beta RPM from 1
>  
>  This idea pushes the limits of my current rpm/yum/dnf foo though, so I
>  figured I'd ask for advice here before I started hacking away at the
>  problem :)
>  
>  Cheers,
>  Nick.
>  
>  --
>  Nick Coghlan
>  Red Hat PnT Operations
>  DevOps Enablement, Brisbane
>  
>  Software Development Workflow Designer & Process Architect
>

[Dan Callaghan]

Excerpts from Nick Coghlan's message of 2015-06-25 15:05 +10:00:
>  With Python 3.5 in beta upstream, I'd like to create a Beaker
> recipe that:
>  
>  1. Rebuilds the Fedora Python RPMs using the upstream Python 3.5 beta
>  tarball rather than the stable 3.4 release 
This will involve tweaking the spec file and making sure it successfully 
builds so you will need to do this step outside Beaker beforehand. Use 
mock to build your hacked SRPM.

There is a fairly scary-looking explanation in python3.spec about how to 
bootstrap the build due to cyclical dependencies with pip so this part 
may be fun...

You can then place the resulting RPMs into a yum repo accessible to your 
recipe and add it to the MOCK_REPOS parameter.

>  2. Queries the Fedora repos to get the list of all packages with a
> build
>  or runtime requirement on Python 3 
Matej has a good suggestion for this.

>  3. Uses
>
> https://beaker-project.org/docs-develop/user-guide/beaker-provided-tasks....
>  to rebuild all the packages from 2 against the Python 3.5 beta RPM from 1 
The example in the docs should work well in this case. The second repo 
in the MOCK_REPOS param will be your yum repo containing the Python 3.5 
RPMs. Instead of SKIP_NOARCH you will want to populate the 
SRPM_WHITELIST parameter based on the list of SRPMs from step 2, bearing 
in mind that the whitelist is a list of globs matched against the entire 
filename including .src.rpm extension.

-- 
Dan Callaghan <dcallagh(a)redhat.com&gt;
Senior Software Engineer, Products & Technologies Operations
Red Hat, Inc.

[Nick Coghlan]

On 06/26/2015 08:40 AM, Dan Callaghan wrote:
>  Excerpts from Nick Coghlan's message of 2015-06-25 15:05 +10:00:
> > With Python 3.5 in beta upstream, I'd like to create a Beaker recipe that:
> >
> > 1. Rebuilds the Fedora Python RPMs using the upstream Python 3.5 beta
> > tarball rather than the stable 3.4 release
>  
>  This will involve tweaking the spec file and making sure it successfully 
>  builds so you will need to do this step outside Beaker beforehand. Use 
>  mock to build your hacked SRPM.
>  
>  There is a fairly scary-looking explanation in python3.spec about how to 
>  bootstrap the build due to cyclical dependencies with pip so this part 
>  may be fun... 
Hmm, maybe I can talk Slavek or Matej into handling that part... on the
other hand, I should probably learn how to do it myself at *some*
point... :)

I've turned the thread into some initial notes in the GitHub repo Matej
created for this:
https://github.com/fedora-python/Rebuild-Beaker-Recipe/wiki

Cheers,
Nick.

-- 
Nick Coghlan
Red Hat PnT Operations
DevOps Enablement, Brisbane

Software Development Workflow Designer & Process Architect