Koji build root caching

[Vít Ondruch]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dne 14.1.2015 v 23:46 Dennis Gilmore napsal(a):
> On Wed, 14 Jan 2015 16:40:35 +0100
> Vít Ondruch <vondruch at redhat.com> wrote:
>
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
>
> > Dne 14.1.2015 v 16:00 Dennis Gilmore napsal(a):
> >> On Wed, 14 Jan 2015 14:57:59 +0100
> >> Miroslav Suchý <msuchy at redhat.com> wrote:
> >>
> >>> On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
> >>>> that all being said. koji doesn't use any caching and will not
> >>>> use the lvm plugin. we make every buildroot from scratch using a
> >>>> fully clean environment to help with ensuring reproducability.
> >>
> >>> You can cache and still preserve reproducability. What I'm
> >>> planning for Copr is to do (every week/month) for chroot in
> >>> fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
> >>>   done
> >>> take snapshot of that. I plan to do that on VM level.
> >>> And when new task come, I will just restore from that snapshot.
> >>> And mock will start with already populated cache. So I will have
> >>> better caching and yet reproducability.
> >>
> >> you really can't.  you would need to make a new cache any time one
> >> of the packages in the minimal buildroot changes.
>
> > Actually this is not anytime. newRepo has to be run, which is not run
> > more then 4 times per hour I'd say. If the new snapshot is prepared as
> > part of newRepo task, the mock could reuse the snapshot.
>
> ok I have a data point here https://ausil.fedorapeople.org/buildroots
> the file in the link is the output of a sql query on koji's db it gives
> the number of times a unique repo was used on a builder to do a build.

Sorry, but would you mind to explain what the columns actually mean?

Thanks.

Vít


> so basically it tells you if we cached the buildroots somehow  how many
> times they would get used.
>
> $ grep "^ 19 |" buildroots|wc -l
> 1
> $ grep "^ 18 |" buildroots|wc -l
> 0
> $ grep "^ 17 |" buildroots|wc -l
> 0
> $ grep "^ 16 |" buildroots|wc -l
> 2
> $ grep "^ 15 |" buildroots|wc -l
> 1
> $ grep "^ 14 |" buildroots|wc -l
> 0
> $ grep "^ 13 |" buildroots|wc -l
> 8
> $ grep "^ 12 |" buildroots|wc -l
> 13
> $ grep "^ 11 |" buildroots|wc -l
> 14
> $ grep "^ 10 |" buildroots|wc -l
> 15
> $ grep "^  9 |" buildroots|wc -l
> 20
> $ grep "^  8 |" buildroots|wc -l
> 24
> $ grep "^  7 |" buildroots|wc -l
> 40
> $ grep "^  6 |" buildroots|wc -l
> 59
> $ grep "^  5 |" buildroots|wc -l
> 84
> $ grep "^  4 |" buildroots|wc -l
> 155
> $ grep "^  3 |" buildroots|wc -l
> 409
> $ grep "^  2 |" buildroots|wc -l
> 1446
> $ grep "^  1 |" buildroots|wc -l
> 4794
>
> What I get from this is that in the majority of cases caching won't
> help at all.  The cost of creating the cache is higher than the benefit
> we would get.
>
> Dennis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJUt9uZAAoJEAzgnueZF7h8NmoP/3igDT3GJBBNDttOMjF2aIKA
y+F9ZyEMorp4nGYN2MK7tcwMC8fdwhBeNNI7OYkJqLru1dhsHszLj1VpOIE5gGaV
gQmoULyDpklDBa0JCa+2kwI3LQGad9m6KjTXo3jhBrOhy+5wJnzxKIdi8GdZzDTL
giHtnxMjt1JncbD0ve7jhx8QPb+x4ZssDG1KuXMOKl7Skz78qPH6ZVGJ3FP1fBnA
4KrBNKRgy2AtB0dqLIvr2m6itUVb2DCNVbqvoIWfKJJM6k+bPt5e90efCsb2ooxD
gNCsjayzRqyMoFSQW+E4sLss+vV0vXajPIsb5eyWhIiEwbTcQQlqGKb1d5PjeVzn
f4tCEVOr5jf0Mx7OLumqdpHPvnPI8ymZt1ZvF3S4SZxy2iiCgR3a40hc0oXoztRd
7LazKk5CKH+W0zhbXMOiKpCk85h8ysFXxcurOBQnRUdGiEgMLxLX5cMfcl41uVko
PAU4n/QAh+6hWoA+ILWglUOrBlmX2XESTSDiUsxfDUpuxIm/amsq/+UD+K3z6EXO
Bv9FbqLb9qVn10kJUDVE7vokBQ75RNGWCvmC3SkyN/gpXX3Ht/whqYTbiQ35iWWo
jN8RGvDvdx4y/WV0RdmqmWphQDMr3K5DEIlKbbknqKyi2s3vk3wge8ApEkOtNVAn
TbtDNyDokF0x6jj6/gy4
=k3fH
-----END PGP SIGNATURE-----