Koji build root caching

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 15 Jan 2015 16:24:09 +0100
Vít Ondruch <vondruch at redhat.com> wrote:

> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dne 14.1.2015 v 23:46 Dennis Gilmore napsal(a):
> > On Wed, 14 Jan 2015 16:40:35 +0100
> > Vít Ondruch <vondruch at redhat.com> wrote:
> >
> >
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> >
> > > Dne 14.1.2015 v 16:00 Dennis Gilmore napsal(a):
> > >> On Wed, 14 Jan 2015 14:57:59 +0100
> > >> Miroslav Suchý <msuchy at redhat.com> wrote:
> > >>
> > >>> On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
> > >>>> that all being said. koji doesn't use any caching and will not
> > >>>> use the lvm plugin. we make every buildroot from scratch using
> > >>>> a fully clean environment to help with ensuring
> > >>>> reproducability.
> > >>
> > >>> You can cache and still preserve reproducability. What I'm
> > >>> planning for Copr is to do (every week/month) for chroot in
> > >>> fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
> > >>>   done
> > >>> take snapshot of that. I plan to do that on VM level.
> > >>> And when new task come, I will just restore from that snapshot.
> > >>> And mock will start with already populated cache. So I will have
> > >>> better caching and yet reproducability.
> > >>
> > >> you really can't.  you would need to make a new cache any time
> > >> one of the packages in the minimal buildroot changes.
> >
> > > Actually this is not anytime. newRepo has to be run, which is not
> > > run more then 4 times per hour I'd say. If the new snapshot is
> > > prepared as part of newRepo task, the mock could reuse the
> > > snapshot.
> >
> > ok I have a data point here
> > https://ausil.fedorapeople.org/buildroots the file in the link is
> > the output of a sql query on koji's db it gives the number of times
> > a unique repo was used on a builder to do a build.
> 
> Sorry, but would you mind to explain what the columns actually mean?
Sure

The first column is the number of times a repo has been used on a
individual builder
The second column is the tag name that populates the buildroot
The third column is the repo id each time koji runs a newRepo task the
resulting repo gets a repo_id that koji uses to identify the repo in
use for a given task. this way we can easily go back to a given repo to
reproduce a build
the fourth column is the host id  each host in koji has a unique id.


the result is that the the greatest bulk of repos are used once or
twice on a given host and that we would get no benefit but a greater
cost in caching minimal buildroots

Dennis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJUuE4sAAoJEH7ltONmPFDRZR8QAL+VCeF1VPI7WSc4Th+ZdvmR
JQrEqJSud5y8gucuTyU2R+sYcLFTaZR2qcjWaPdzQS7XiIIhv68KlKbganq6/r4g
bHEytdcG9mZXxV69Uojq7hbfpDvd0EJu6sKCAy/uz2fuOJt/mdvxld5nVIpHBvJH
vd8j+XN6jFOjb3FTVCrW/gxXEKW2+1eA7XsECu+e7XS1rBNC04FkFlZEbK9thMoQ
K0ICXhrOtLXaara7ZuC7afP+xBvuTHjUQiFWrUDPrCtPKZXiXjXfk0rRFfhY+VHO
ZJP7OR/RkLQ3dDVgRnHsVTxu5He0XdVnrsGmnPBneS+ThH46HEfVTMOjs4dUVcbW
EjSzsYlPhiHCBMOi21NPgBH0XhB6x/9mCi6NYUTCFHkP8OHLQTVtcbA37gp6weqh
0UEhHtnmPCC/cD0UbmOhRD4ac75RQAMLt8vQr98g2HHzsis30gW1/A6KlBxZvhe5
d0ESfQEp3o/ex1wsw/b3oVuI9fPSoYH/c9nU8yEqZhx9bP8CoSXmvgHxxRBooZgf
/7Uc+jANIJx0vFL8BPPPOhgGhOGe1VsJpUZ4gIH5Ee+vmXMqQOa4rOcWTLlmUjUD
t60DuORK0gQdiXh6sljpXnyDYcJXleDXqq89hztdnCtaN+pUQsPhzawTnibjDlTL
1mGTikF3qAVK2EPJtOTN
=I42g
-----END PGP SIGNATURE-----