Remove gcc, gcc-c++ and make from minimal build root
Dennis Gilmore
dennis at ausil.us
Tue Jan 13 17:01:45 UTC 2015
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Tue, 13 Jan 2015 10:28:36 -0500
Josh Boyer <jwboyer at fedoraproject.org> wrote:
> On Tue, Jan 13, 2015 at 9:56 AM, Vít Ondruch <vondruch at redhat.com>
> wrote:
> > Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):
> >
> > On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
> >
> > On Tue, Jan 13, 2015 at 12:16:42PM +0000, Richard W.M. Jones wrote:
> >
> > That's simply an idiotic thing to say. What is the progress
> > involved in adding a new BR to thousands of packages?
> >
> > Am I wrong assuming that it would be rather easy to add the BR to
> > all packages semi-automatically previous to a mass rebuild so that
> > people who want to get rid of gcc for their packages can easily
> > remove it?
> >
> > What am I missing?
> >
> > What is missing is there is no quantifcation yet of the benefit.
> > How much faster will builds be?
> >
> >
> > Since you are asking I did 5 times:
> >
> > $ mock -r minimal --scrub all && time mock -r minimal --init
> >
> > Where the "Total" is DNF's output of package download speed and
> > amount of downloaded data obtained from mock output. Other DNF
> > information are shown on top of each section as well.
> >
> >
> > == with current minimal build root, i.e. "install @buildsys-build"
> >
> > Install 164 Packages
> >
> > Total download size: 119 M
> > Installed size: 445 M
> >
> > 1)
> > Total 26 MB/s | 119
> > MB 00:04 real 0m51.364s
> > user 0m33.050s
> > sys 0m6.454s
> >
> > 2)
> > Total 13 MB/s | 119
> > MB 00:09 real 0m51.479s
> > user 0m33.024s
> > sys 0m3.400s
> >
> > 3)
> > Total 19 MB/s | 119
> > MB 00:06 real 0m52.373s
> > user 0m33.171s
> > sys 0m7.121s
> >
> > 4)
> > Total 21 MB/s | 119
> > MB 00:05 real 0m54.213s
> > user 0m32.999s
> > sys 0m7.122s
> >
> > 5)
> > Total 13 MB/s | 119
> > MB 00:09 real 0m56.093s
> > user 0m32.708s
> > sys 0m5.745s
> >
> >
> > == without gcc, gcc-c++ and make, i.e. "install rpm-build
> > shadow-utils util-linux which"
> >
> > Install 153 Packages
> >
> > Total download size: 81 M
> > Installed size: 350 M
> >
> > 1)
> > Total 14 MB/s | 81
> > MB 00:05 real 0m49.789s
> > user 0m28.926s
> > sys 0m6.864s
> >
> > 2)
> > Total 17 MB/s | 81
> > MB 00:04 real 0m44.010s
> > user 0m28.965s
> > sys 0m3.434s
> >
> > 3)
> > Total 18 MB/s | 81
> > MB 00:04 real 0m45.860s
> > user 0m29.032s
> > sys 0m5.730s
> >
> > 4)
> > Total 19 MB/s | 81
> > MB 00:04 real 0m42.047s
> > user 0m28.944s
> > sys 0m3.099s
> >
> > 5)
> > Total 14 MB/s | 81
> > MB 00:05 real 0m47.079s
> > user 0m29.839s
> > sys 0m3.978s
> >
> >
> > == without gcc, gcc-c++, make and without perl (rhbz#1158860, note
> > that the rpm packages are local in this case), i.e. "install
> > rpm-build shadow-utils util-linux which"
> >
> > Install 124 Packages
> >
> > Total size: 69 M
> > Total download size: 68 M
> > Installed size: 310 M
> >
> > 1)
> > Total 11 MB/s | 68
> > MB 00:06 real 0m44.725s
> > user 0m27.644s
> > sys 0m6.126s
> >
> > 2)
> > Total 12 MB/s | 68
> > MB 00:05 real 0m46.253s
> > user 0m27.624s
> > sys 0m7.056s
> >
> > 3)
> > Total 6.5 MB/s | 68
> > MB 00:10 real 0m50.296s
> > user 0m27.567s
> > sys 0m6.064s
> >
> > 4)
> > Total 12 MB/s | 68
> > MB 00:05 real 0m46.078s
> > user 0m27.581s
> > sys 0m7.698s
> >
> > 5)
> > Total 16 MB/s | 68
> > MB 00:04 real 0m44.307s
> > user 0m27.830s
> > sys 0m5.293s
> >
> > == Only rpmbuild install "install rpm-build shadow-utils"
> >
> > Install 111 Packages
> >
> > Total size: 61 M
> > Total download size: 60 M
> > Installed size: 286 M
> >
> > 1)
> > Total 20 MB/s | 60
> > MB 00:03 real 0m37.848s
> > user 0m26.158s
> > sys 0m2.676s
> >
> > 2)
> > Total 13 MB/s | 60
> > MB 00:04 real 0m44.756s
> > user 0m26.442s
> > sys 0m5.185s
> >
> > 3)
> > Total 16 MB/s | 60
> > MB 00:03 real 0m42.703s
> > user 0m26.118s
> > sys 0m3.174s
> >
> > 4)
> > Total 13 MB/s | 60
> > MB 00:04 real 0m42.365s
> > user 0m26.439s
> > sys 0m4.193s
> >
> > 5)
> > Total 5.9 MB/s | 60
> > MB 00:10 real 0m48.940s
> > user 0m26.608s
> > sys 0m4.576s
> >
> > == Minimal root I can shell in, i.e. "install bash shadow-utils"
> >
> > Install 48 Packages
> >
> > Total download size: 31 M
> > Installed size: 176 M
> >
> > 1)
> > Total 8.3 MB/s | 31
> > MB 00:03 real 0m36.745s
> > user 0m22.019s
> > sys 0m4.936s
> >
> > 2)
> > Total 9.5 MB/s | 31
> > MB 00:03 real 0m32.608s
> > user 0m21.805s
> > sys 0m2.719s
> >
> > 3)
> > Total 11 MB/s | 31
> > MB 00:02 real 0m34.111s
> > user 0m22.170s
> > sys 0m3.651s
> >
> > 4)
> > Total 12 MB/s | 31
> > MB 00:02 real 0m41.141s
> > user 0m22.595s
> > sys 0m5.535s
> >
> > 5)
> > Total 8.4 MB/s | 31
> > MB 00:03 real 0m34.303s
> > user 0m21.851s
> > sys 0m3.839s
> >
> >
> > And several times during the test I encountered bug similar to:
> >
> > Downloading Packages:
> > Error: Error downloading packages:
> > Curl error: Failure when receiving data from the peer for
> > ftp://mirror.slu.cz/fedora/linux/development/rawhide/x86_64/os/Packages/e/elfutils-libelf-0.161-1.fc22.x86_64.rpm
> > [response reading failed]
> >
> > Less packages to download, less possible issues.
> >
> >
> > You can interpret these data yourself, but with less packages in
> > build root, I can see:
> >
> > 1) Saved build time
> > 2) Saved network bandwidth
> > 3) Saved storage
> > 4) Less things to break
3 is not a savings we would see
1 and 2 would be on some unknown number of builds
as to 4 we have breakages yes. but they are not that common. so while
it will be some saving at build time. It could turn into bigger issues
elsewhere if those components get less testing and breakages not
notcied, which is practice is likely not going to happen. There is too
much pulling at straws in all the arguments I have seen. lets get some
concrete data to look at.
> Just to make sure I'm reading the data correctly, your tests show a
> maximum approximate savings of 20 seconds going from the current
> buildroot to the minimal shell root?
>
> > Versus how long will it take to
> > either modify all these packages / write and run the script to do
> > that? Is A > B?
> >
> >
> > There are things which are hard to quantify. If you want to
> > prohibit any change, this is probably right question to ask. Of
> > course answering such questions also takes some time, I hope you
> > will count it into the final equation at the end, although hard to
> > say on which side it should belong.
>
> I don't think the original question was all that great. If we were to
> script the change to add the BRs, it would take time but it would
> mostly be a one-time cost. Reducing buildroot time creation, network
> bandwidth, and storage savings is a reduction over each new buildroot
> instance creation over a long period of time. And as you said, you
> could also wind up saving time for issues where the buildroot doesn't
> actually install because of broken packages (e.g. a broken perl dep or
> something). The question, as posed, wasn't really taking everything
> into account.
>
> That being said, 20 seconds isn't saving very much. The network
> bandwidth savings exists, but given that mock caches packages it isn't
> a savings for every build. The storage savings I'm not sure I really
> agree with because mock roots are transient things and you can delete
> them if you need to. You only wind up saving a few megabytes. I'm
> not sure this is compelling enough data at the moment.
that all being said. koji doesn't use any caching and will not use the
lvm plugin. we make every buildroot from scratch using a fully clean
environment to help with ensuring reproducability. the only real win
here is for things not using gcc or gcc-c++ what % of packages are
they? what savings will they really get in faster builds? i.e. are the
extra things pulled in by gcc and gcc-c++ pulled in by other things
those packages need. how much time will we spend fixing build issues
due to the fallout from making this change. some of these we can not
work out easily but some we can.
Dennis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBAgAGBQJUtU95AAoJEH7ltONmPFDR/9oP/23p7LgMoU7bnfGrJP4zSwSI
nRfhZBg3LdCrI5MMihZch6zVyhHde0nH6+6h5PSMx9YbhJCTUw3wzCOuIpn0tysT
u35HiTDn27V7QNfN554CIGAWtM6S2jMABex4XdTPm/oKlgZ2V1OCFL4t+u9+hzfN
gu4OCPt1fxpjVmAWoaE1hd6jB/pOjDqASygHP/puE6T6Lo+lyx7HdRdu+9l46Mgm
1r9kuTzdb7CmkzNLomf3zMMdN+4hZ+7nomjmfvjCikaMCp3sO9qNlLw8EAONmNkv
ZaupCfxPslnmvtJ4F/Q23+9W01jzyssXHNkzDiw3KIyYJMOdYQzv5dBd/HViK06l
lBjtSnbakghJF6he38AzqxIVilVXYXt+t7DIcFeysmYMihgok3JFgjXrM6afh7Mg
nnV5jG2FIKpkbRoVNaNjHCX2c03jA7v6KLnvdn4JQUZ6JoJxH/rvY07wMpphUzl3
wvQjYGwjHgdbDWC/bDK637VcBFQvBPmzaLC/pTJSUDaJoNNIOAm2+XivSsgOG9dA
qRqh5jSpkzdamzZsFabrM4ikFCGFsfEbsM5KHFC6lLigDqi9o9CYpBjCV/OHEaZu
Q+6TaedU/tGhROqrKj5Yiw+JZkHZJC7AhA8AY8BPCbXhcLQyLoz3gf4IqFFOIfVB
aPlaJ2P7Pg0Ez3g6+ptO
=rYA/
-----END PGP SIGNATURE-----
More information about the devel
mailing list