Make buildSRPMFromSCM faster?

[Richard W.M. Jones]

On Mon, Jul 21, 2014 at 01:58:49PM -0500, Dennis Gilmore wrote:
> On Sat, 19 Jul 2014 12:30:38 +0100
> "Richard W.M. Jones" <rjones at redhat.com> wrote:
> 
> > 
> > The first step of most Koji builds is buildSRPMFromSCM, where a
> > .src.rpm file is built from the git repo.
> > 
> > Currently this involves completely building a mock buildroot
> > containing all the BuildRequires, and running `rpmbuild -bs'.  This
> > takes many minutes (especially when arm is chosen as a builder).
> > 
> > It seems the reason for this is because the spec file has to be fully
> > parsed in order to work out the Source lines.  Since Source lines
> > might depend on RPM macros which might depend on any BuildRequire'd
> > package, every BR package must be installed in the mock root.
> > `rpmbuild -bs' takes seconds because it just bundles all the Source
> > files with the spec file into an SRPM.
> > 
> > Is this really necessary?
> This is not at all true. we have a minimal buildroot that is installed
> for every single build, the buildroot packages are the same across all
> builds. there is no BuildRequires pulled in until the buildArch task

Right, see my other reply.  However it still takes 6 minutes to build
the chroot.

> > Two shortcuts seem possible:
> > 
> > (1) Limit the use of macros in Source lines, so that only a simple,
> > standard, perhaps pre-cached buildroot can be used.
> > 
> > (2) Perhaps uglier: Just build an SRPM that contains everything in
> > dist-git + everything in the lookaside cache, and hope for the
> > best ...
>
> This is what we do now.

My suggestion with (2) is that we shortcut building the .src.rpm by
taking everything in dist-git + the files mentioned in sources (from
the lookaside cache), then build something that looks like a .src.rpm.
I doubt you can actually do this using current rpm tools.  This should
be very quick - seconds not minutes.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v