I'm starting to build Docker images on the Fedora base and I'm a little concerned about the size of the images. For example, the Debian base is 85.1 MB, the Ubuntu base is 192 MB and the CentOS base is 224 MB. The Fedora base is 250.2 MB and that's *before* I install any application code.
Is there any way I can build a smaller base? Ideally I'd like something under 100 MB but even getting it under 200 MB would be worthwhile.
On Thu, Jan 08, 2015 at 11:36:04PM -0800, M. Edward (Ed) Borasky wrote:
I'm starting to build Docker images on the Fedora base and I'm a little concerned about the size of the images. For example, the Debian base is 85.1 MB, the Ubuntu base is 192 MB and the CentOS base is 224 MB. The Fedora base is 250.2 MB and that's *before* I install any application code. Is there any way I can build a smaller base? Ideally I'd like something under 100 MB but even getting it under 200 MB would be worthwhile.
Yeah it seems to have crept up a bit. The image went from this Cloud SIG over to the Base WG (since there's so much interest in Docker), but I think we still have a vested interest in keeping it small. In general, much of the regular dependency-shrinking work we've been doing should help here.
Unfortunately, the easy work is mostly done, as is some of the moderately hard. The next most helpful thing is probably making a compact version of python sufficient for yum, cloud-init, grubby (new version), and _possibly_ firewalld. As it is, it's about a third
Oh hmmm. I see that grub2 got into the docker image somehow. That should probaby be fixed, and provides a big savings even in the compressed version because that's the only thing that pulls in fedora-logos.
Hmmm. Firewalld too. That really doesn't need to be there, does it?
That said, the docker image we're producing *is* only 90MB compressed — see https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Docker/x86_64/ — and 241MB uncompressed. Something _additionally_ weird is going on.
Are you interested in taking point on this for Fedora 22? Or finding a handful of people who are interested in working with the Base WG to figure it out?
On Tue, Jan 13, 2015 at 3:18 PM, Matthew Miller mattdm@fedoraproject.org wrote:
On Thu, Jan 08, 2015 at 11:36:04PM -0800, M. Edward (Ed) Borasky wrote:
I'm starting to build Docker images on the Fedora base and I'm a little concerned about the size of the images. For example, the Debian base is 85.1 MB, the Ubuntu base is 192 MB and the CentOS base is 224 MB. The Fedora base is 250.2 MB and that's *before* I install any application code. Is there any way I can build a smaller base? Ideally I'd like something under 100 MB but even getting it under 200 MB would be worthwhile.
Yeah it seems to have crept up a bit. The image went from this Cloud SIG over to the Base WG (since there's so much interest in Docker), but I think we still have a vested interest in keeping it small. In general, much of the regular dependency-shrinking work we've been doing should help here.
Unfortunately, the easy work is mostly done, as is some of the moderately hard. The next most helpful thing is probably making a compact version of python sufficient for yum, cloud-init, grubby (new version), and _possibly_ firewalld. As it is, it's about a third
Oh hmmm. I see that grub2 got into the docker image somehow. That should probaby be fixed, and provides a big savings even in the compressed version because that's the only thing that pulls in fedora-logos.
Hmmm. Firewalld too. That really doesn't need to be there, does it?
That said, the docker image we're producing *is* only 90MB compressed — see https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Docker/x86_64/ — and 241MB uncompressed. Something _additionally_ weird is going on.
Are you interested in taking point on this for Fedora 22? Or finding a handful of people who are interested in working with the Base WG to figure it out?
The list from the latest image in the docker repository is: http://paste.fedoraproject.org/168966/21139532/
I was actually playing with docker today and was poking at it. Happy to help push fixes and help generally too.
Peter
On Tue, Jan 13, 2015 at 03:28:14PM +0000, Peter Robinson wrote:
The list from the latest image in the docker repository is: http://paste.fedoraproject.org/168966/21139532/
Same list, sorted by package size (rpm -qa --qf'%9{size} %{name}\n'|sort -rn)
http://paste.fedoraproject.org/169170/42116857
(But note that glibc-common is deceptive, because without a better technical approach available right now, RPM doesn't properly understand that the translations have not been installed.)
On 13/01/15 17:07, Matthew Miller wrote:
On Tue, Jan 13, 2015 at 03:28:14PM +0000, Peter Robinson wrote:
The list from the latest image in the docker repository is: http://paste.fedoraproject.org/168966/21139532/
Same list, sorted by package size (rpm -qa --qf'%9{size} %{name}\n'|sort -rn)
http://paste.fedoraproject.org/169170/42116857
(But note that glibc-common is deceptive, because without a better technical approach available right now, RPM doesn't properly understand that the translations have not been installed.)
Note coreutils is up the top at 14.6MB It has a new feature in recent versions to reduce disk usage with: ./configure --enable-single-binary It might be worth trying out? Caveat is some more shared libs for some tools, which associated higher virt mem usage. This drops the binaries disk usage from about 5.3MB to 1.2MB (there are about 10MB of translations and docs).
cheers, Pádraig.
On Tue, Jan 13, 2015 at 6:48 PM, Pádraig Brady P@draigbrady.com wrote:
Note coreutils is up the top at 14.6MB It has a new feature in recent versions to reduce disk usage with: ./configure --enable-single-binary It might be worth trying out? Caveat is some more shared libs for some tools, which associated higher virt mem usage. This drops the binaries disk usage from about 5.3MB to 1.2MB (there are about 10MB of translations and docs).
cheers, Pádraig.
The images I've been looking at have a lot of "translations and docs" removed compared to a stock Fedora Server. I assume those could be removed from coreutils as well.
cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
What's the process? I found grub2 and fedora-logos in my own testing but didn't know about firewalld? Does one simply remove things and run a test suite? *Is* there a test suite?
How about dracut? Does that need to be there?
On Tue, Jan 13, 2015 at 7:18 AM, Matthew Miller mattdm@fedoraproject.org wrote:
On Thu, Jan 08, 2015 at 11:36:04PM -0800, M. Edward (Ed) Borasky wrote:
I'm starting to build Docker images on the Fedora base and I'm a little concerned about the size of the images. For example, the Debian base is 85.1 MB, the Ubuntu base is 192 MB and the CentOS base is 224 MB. The Fedora base is 250.2 MB and that's *before* I install any application code. Is there any way I can build a smaller base? Ideally I'd like something under 100 MB but even getting it under 200 MB would be worthwhile.
Yeah it seems to have crept up a bit. The image went from this Cloud SIG over to the Base WG (since there's so much interest in Docker), but I think we still have a vested interest in keeping it small. In general, much of the regular dependency-shrinking work we've been doing should help here.
Unfortunately, the easy work is mostly done, as is some of the moderately hard. The next most helpful thing is probably making a compact version of python sufficient for yum, cloud-init, grubby (new version), and _possibly_ firewalld. As it is, it's about a third
Oh hmmm. I see that grub2 got into the docker image somehow. That should probaby be fixed, and provides a big savings even in the compressed version because that's the only thing that pulls in fedora-logos.
Hmmm. Firewalld too. That really doesn't need to be there, does it?
That said, the docker image we're producing *is* only 90MB compressed — see https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Docker/x86_64/ — and 241MB uncompressed. Something _additionally_ weird is going on.
Are you interested in taking point on this for Fedora 22? Or finding a handful of people who are interested in working with the Base WG to figure it out?
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader _______________________________________________ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
On Tue, Jan 13, 2015 at 11:02:34AM -0800, M. Edward (Ed) Borasky wrote:
What's the process? I found grub2 and fedora-logos in my own testing but didn't know about firewalld? Does one simply remove things and run a test suite? *Is* there a test suite?
No test suite yet. Help wanted. :)
It's made from this kickstart — https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-docker-bas...
and fed through ImageFactory in Koji to produce a tarball of / as the output.
How about dracut? Does that need to be there?
It seems very unlikely.
On Tue, Jan 13, 2015 at 1:13 PM, Matthew Miller mattdm@fedoraproject.org wrote:
On Tue, Jan 13, 2015 at 11:02:34AM -0800, M. Edward (Ed) Borasky wrote:
What's the process? I found grub2 and fedora-logos in my own testing but didn't know about firewalld? Does one simply remove things and run a test suite? *Is* there a test suite?
No test suite yet. Help wanted. :)
Right now I have only two Fedora-based Docker images - https://registry.hub.docker.com/u/znmeb/osjourno-rd/ is the most active and the test suite is
a. Make sure it builds on Docker Hub b. docker run c. Browse to the RStudio Server and make all the documents from templates (by hand).
I can probably automate the third step via Firefox but I haven't taken the time to do that.
I am planning to integrate it with a PostgreSQL/PostGIS image and a Redis image in the near future but I may use the "official" images from Docker Hub (on a Debian base) rather than the ones in "fedora-dockerfiles"..
It's made from this kickstart — https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-docker-bas...
and fed through ImageFactory in Koji to produce a tarball of / as the output.
Given a working Docker image, is there some combination of 'docker export', tar archive manipulation and 'docker import' that can achieve the same effect? I tried just piping an export with packages removed to an import, but that didn't make the image smaller.
How about dracut? Does that need to be there?
It seems very unlikely.
BTW, this conversation is fractured between this mailing list thread and the TRAC issue #66. We should merge the threads somehow.
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader _______________________________________________ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
I have some preliminary results in my OSJourno project (https://bitbucket.org/znmeb/osjourno/src/fc32383d24137484d8b3fa2b7bff3afe436...)
1. I couldn't find all the missing pieces for Image Factory (there's a template file needed) so I did the build using livemedia-creator. It took a couple of edits to get a working kickstart but that's in osjourno-docker-base.ks.
2. If you remove kpartx, grub2, dracut and firewalld, systemd is missing from the image, which errors the livemedia-creator install and probably yields a non-functional Fedora anyhow. So I added systemd to the kickstart.
3. The resulting image builds znmeb/osjourno-rd correctly and the few run-time tests I did worked.
4. Sizes: original Fedora base: 250.2 MB, reduced image: 179.9 MB. That's a saving of 78.3 MB and makes the Fedora image smaller than the Ubuntu one.
On Tue, Jan 13, 2015 at 1:32 PM, M. Edward (Ed) Borasky znmeb@znmeb.net wrote:
On Tue, Jan 13, 2015 at 1:13 PM, Matthew Miller mattdm@fedoraproject.org wrote:
On Tue, Jan 13, 2015 at 11:02:34AM -0800, M. Edward (Ed) Borasky wrote:
What's the process? I found grub2 and fedora-logos in my own testing but didn't know about firewalld? Does one simply remove things and run a test suite? *Is* there a test suite?
No test suite yet. Help wanted. :)
Right now I have only two Fedora-based Docker images - https://registry.hub.docker.com/u/znmeb/osjourno-rd/ is the most active and the test suite is
a. Make sure it builds on Docker Hub b. docker run c. Browse to the RStudio Server and make all the documents from templates (by hand).
I can probably automate the third step via Firefox but I haven't taken the time to do that.
I am planning to integrate it with a PostgreSQL/PostGIS image and a Redis image in the near future but I may use the "official" images from Docker Hub (on a Debian base) rather than the ones in "fedora-dockerfiles"..
It's made from this kickstart — https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-docker-bas...
and fed through ImageFactory in Koji to produce a tarball of / as the output.
Given a working Docker image, is there some combination of 'docker export', tar archive manipulation and 'docker import' that can achieve the same effect? I tried just piping an export with packages removed to an import, but that didn't make the image smaller.
How about dracut? Does that need to be there?
It seems very unlikely.
BTW, this conversation is fractured between this mailing list thread and the TRAC issue #66. We should merge the threads somehow.
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader _______________________________________________ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
-- OSJourno: Robust Power Tools for Digital Journalists http://www.znmeb.mobi/stories/osjourno-robust-power-tools-for-digital-journa...
Remember, if you're traveling to Bactria, Hump Day is Tuesday and Thursday.
-
M. Edward (Ed) Borasky -
Matthew Miller -
Peter Robinson -
Pádraig Brady