Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, 17 Jan 2015 03:44:10 +0000
Peter Robinson <pbrobinson at gmail.com> wrote:

> >> >>> > > that all being said. koji doesn't use any caching and will
> >> >>> > > not use the lvm plugin. we make every buildroot from
> >> >>> > > scratch using a fully clean environment to help with
> >> >>> > > ensuring reproducability.
> >> >> >
> >> >> > You can cache and still preserve reproducability. What I'm
> >> >> > planning for Copr is to do (every week/month) for chroot in
> >> >> > fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
> >> >> >   done
> >> >> > take snapshot of that. I plan to do that on VM level.
> >> >> > And when new task come, I will just restore from that
> >> >> > snapshot. And mock will start with already populated cache.
> >> >> > So I will have better caching and yet reproducability.
> >> > you really can't.  you would need to make a new cache any time
> >> > one of the packages in the minimal buildroot changes.
> >>
> >> That is why mock run 'yum update' on minimal buildroot before
> >> proceeding further. So if one package in buildroot changes, you
> >> will just download and install just that one package and all other
> >> comes from cache.
> >
> > That invalidates being able to reproduce the build root exactly. as
> > you would need to know and reproduce the package update. This is
> > something that has undergone a lot of thought and consideration and
> > there is very very good reasons things are the way they are. as i
> > showed elsewhere in this thread most repos are used only once or
> > twice at most on any given host.
> 
> The only way you could cache a base build root effectively is if you
> track the packages and NVRs and when one of them bump you regenerate
> the buildroot, likely pausing all builds until a new one is there.
> there's likely not that much churn in the packages contained in the
> root buildroot (you'd want to make sure you had everything for
> building .src.rpms too) and in some cases you could likely get days on
> the same image without having to rebuild it, other days you might have
> to do it a few times a time.

As I showed earlier in the thread most repos are only ever used once or
twice on a host. but we do not know if something in the minimal
buildroot has changed. there is a lot of added complexity needed to
attempt to use caching. which adds many more chances for something to
go wrong. I honestly do not know if its quicker to build the minimal
buildroot from scratch or to unpack the tarball. there is costs both
ways. with the added complexity in tracking things to try and make
caching worthwhile.  in the Fedora case we would need to have somewhere
in the order of 6-12 cached buildroots on each builder depending on
sidetags etc in use at different times.  but there is instances of koji
where that number would be in the hundreds. we need to make sure that
it works and scales not only for our use cases but also other users of
koji.

> Whether the effort to implement out weights the effort remains to be
> seen. I think, given the low IOPs on the builds with an underlying COP
> image it might well give a reasonable reduction in building especially
> during mass rebuilds where the builders are very active and as it's in
> a side tag the new build root packages don't end up in the build root
> until tagged. Using fedmsg it's likely not hard either to monitor a
> package subset for changes and regen the image.

There is a massive effort to figure out if the effort to implement any
caching is worth while. If someone wanted to take the time to figure
that out, making sure to gather the needs of different koji users to
make sure that we did not make things worse for them, then by all means
that person is welcome to do so and I will gladly answer any and all
questions that they had. However I have no interest in going down that
rabbit hole. we can not rely on koji users using fedmsg so that is
something that would likely need to be ruled out of the design
decision. fedmsg also does not guarantee delivery of messages and so
can not be relied on for anything where we have to be 100% sure that
things are done given a trigger.

Dennis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJUupbqAAoJEH7ltONmPFDR+iQP/ietNky5y2yhtbUZ/MMGP1er
PDMyHz5MD9v78mstXXjx6WsBrOjc2a51WCufBV2drlPuwyTrstwJShbOhm1XfOld
1HSI9etIkwsVxBFgRfh6HUxbLGZWSHDZviY34KocxLNuuaRLqc5RHjTTZmzQwKl+
4EA4o6HJAYPsVz0Fu2/sJAy9cCclfCPy8Mcpstd1NIHOPhtNTpFiEQBl4zpXRqQs
nlmlNh1MhZepv84k+EvhRZCJlmfn0yLBLlNJfNh6jtdy6zv+lNf2oClrHDR9o5aL
nRCEJhRl/LX+BNf2mV5MULxAliXMME1qoYrm//BgsQk45t3XvVJ2fpPEVs8gcr49
gZ0yoJ20fcSW/UJCRMhoI9dtp1MYQmF3h3LHfNRROv+Ga9Uumw9OmQyFqFJwbF2o
doLOvDxtpEXWsiXcTUSqcl3CBGyT50AiAbRvv5yrQ5TU1Z8TplcUVUbJ/zifBoid
GexFUETsN3E1P0byvfbEiJJgaAYkWTaho2SwO+gRY/iieHlWyEbK6BwAiw7BsJOM
Tm5P6EjHyqk1kPiycoG9Zz70iiBhrFraMgzWwtrjNid2n8vvV6pdEpXpiaDzOAgS
SdVpHH2R8tu8GjnbONazSt6GpG7oEqnS5Nz2YZT6MMTcNuCvxY5bVoRfAl4DlDTM
X/98RH2w7xXM/fejAZN3
=Vjxf
-----END PGP SIGNATURE-----