docker-rpm-builder

[Alan Franzoni]

Hello,
if anybody is interested I've started a very small project which builds
rpms using docker:

https://github.com/alanfranz/docker-rpm-builder

It's a very small wrapper around rpmbuild, and I hope it's a bit more
hackable than mock (which is got plenty of functions but I find it a bit
hard to understand); it's designed to work on any host that support docker
( I had some issues in using mock on host distributions that weren't either
Fedora, Redhat or Centos ).

In our scenarios is already proving faster than mock, but I'm sure there's
plenty of space for optimization.

If anybody is interested in contributing just drop me an email and/or
submit a pull request.

bye,

Alan

-- 
My development blog: http://ollivander.franzoni.eu
contact me at public(a)[mysurname].eu

[Thomas Guthmann]

Hi,

>  https://github.com/alanfranz/docker-rpm-builder 
I had a thought
about it but never found the time to look more into koi 
internals to see how we can hook it.

It would be beneficial in time to replace mock by docker (or plain LXC) 
in koji. But maybe someone more experienced with mock could finger point 
the advantages of mock compare to containers ?

I like koji but our devs hate the time it takes to build RPMs. I have to 
say half of the time is taken by mock deploying the buildroot and our 
builders don't have the luxury to be on SSDs ;)

Thomas

[Alan Franzoni]

Hello Thomas,
our experience with koji showed that we used very little of its
functionality but we took a large performance hit from using it; hence we
first started using mock and a manually built repository creation system,
now we're replacing mock with this docker-rpm-builder.

I'll do some benchmarking for mock-vs-drb later on.

bye,



On Mon, Mar 17, 2014 at 2:57 AM, Thomas Guthmann <tguthmann(a)iseek.com.au&gt;wrote:

>  Hi,
>
>   https://github.com/alanfranz/docker-rpm-builder
> >
>  I had a thought about it but never found the time to look more into koi
>  internals to see how we can hook it.
>
>  It would be beneficial in time to replace mock by docker (or plain LXC) in
>  koji. But maybe someone more experienced with mock could finger point the
>  advantages of mock compare to containers ?
>
>  I like koji but our devs hate the time it takes to build RPMs. I have to
>  say half of the time is taken by mock deploying the buildroot and our
>  builders don't have the luxury to be on SSDs ;)
>
>  Thomas
>
>
>
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys 



-- 
Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
contact me at public(a)[mysurname].eu

[Bohuslav Kabrda]

----- Original Message -----

>  Hello Thomas,
>  our experience with koji showed that we used very little of its functionality
>  but we took a large performance hit from using it; hence we first started
>  using mock and a manually built repository creation system, now we're
>  replacing mock with this docker-rpm-builder. 
>  I'll do some benchmarking for mock-vs-drb later on. 
>  bye, 
Hi, 
I haven't really been digging very deeply into this, but how do you make sure that you
have the most up-to-date buildroot in the Docker image you use? If you're building for
e.g. rawhide, you'd basically need to rebuild the image every time a package from
minimal buildroot gets version/release bump. So IIUC you'd either have to rebuild the
base image pretty often or you'd have to do "yum update" inside the docker
image before you actually start building. At that point, I'm not really sure how much
faster docker is. 
Also, IIUC you require "yum" to be present in the docker image, which will
probably draw in some dependencies that are unnecessary and may be unwanted (e.g. for
Fedora builds). 

Please correct me if I'm wrong. 
Thanks, 
Slavek. 

>  On Mon, Mar 17, 2014 at 2:57 AM, Thomas Guthmann <
> tguthmann(a)iseek.com.au >
>  wrote: 
>  > Hi,
>   
>  > > https://github.com/alanfranz/docker-rpm-builder
>  > 
>   
>  > I had a thought about it but never found the time to look more
> into koi
>  > internals to see how we can hook it.
>   
>  > It would be beneficial in time to replace mock by docker (or
> plain LXC) in
>  > koji. But maybe someone more experienced with mock could finger point the
>  > advantages of mock compare to containers ?
>   
>  > I like koji but our devs hate the time it takes to build RPMs. I
> have to
>  > say
>  > half of the time is taken by mock deploying the buildroot and our builders
>  > don't have the luxury to be on SSDs ;)
>   
>  > Thomas
>   
>  > --
>  
>  > buildsys mailing list
>  
>  > buildsys(a)lists.fedoraproject.org
>  
>  > https://admin.fedoraproject.org/mailman/listinfo/buildsys
>   
>  --
>  Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
>  contact me at public(a)[mysurname].eu 
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys 
-- 
Regards, 
Bohuslav "Slavek" Kabrda.

[Alan Franzoni]

Hello Bohuslav,
currently we don't ensure we've got all the very latest deps, just because
we don't need them - such docker image is used for *building*, not for
deployment. It isn't usually a problem to build against a slightly outdated
library version (unless you're developing a distribution - see later)

The image is rebuilt through docker.io trusted builds service about every
month, just to be sure our image is not too old and "stale".

Our main target, by the way, is building our packages for *stable*
Rhel/Centos releases; if you're building for a development release you'll
probably want to have fully updated deps every time, so you can test
against the very latest headers/libraries.

OTOH if your aim is different, docker-rpm-builder can be hacked however you
like: this is the script that gets executed inside the docker image:

https://github.com/alanfranz/docker-rpm-builder/blob/master/rpmbuild-in-d...

Just add a "yum -y update" before yum-builddep and you'll always get an
up-to-date buildroot. Of course it will be a bit slower, so you'd better
rebuild your image (maybe daily?) on your build machine if speed is
important.

Alan

On Thu, Mar 20, 2014 at 1:59 PM, Bohuslav Kabrda <bkabrda(a)redhat.com&gt; wrote:

>  ------------------------------
>
>  Hello Thomas,
>  our experience with koji showed that we used very little of its
>  functionality but we took a large performance hit from using it; hence we
>  first started using mock and a manually built repository creation system,
>  now we're replacing mock with this docker-rpm-builder.
>
>  I'll do some benchmarking for mock-vs-drb later on.
>
>  bye,
>
>  Hi,
>  I haven't really been digging very deeply into this, but how do you make
>  sure that you have the most up-to-date buildroot in the Docker image you
>  use? If you're building for e.g. rawhide, you'd basically need to rebuild
>  the image every time a package from minimal buildroot gets version/release
>  bump. So IIUC you'd either have to rebuild the base image pretty often or
>  you'd have to do "yum update" inside the docker image before you actually
>  start building. At that point, I'm not really sure how much faster docker
>  is.
>  Also, IIUC you require "yum" to be present in the docker image, which will
>  probably draw in some dependencies that are unnecessary and may be unwanted
>  (e.g. for Fedora builds).
>
>  Please correct me if I'm wrong.
>  Thanks,
>  Slavek.
>
>
>
>  On Mon, Mar 17, 2014 at 2:57 AM, Thomas Guthmann <tguthmann(a)iseek.com.au&gt;wrote:
>
> > Hi,
> >
> > https://github.com/alanfranz/docker-rpm-builder
> >>
> > I had a thought about it but never found the time to look more into koi
> > internals to see how we can hook it.
> >
> > It would be beneficial in time to replace mock by docker (or plain LXC)
> > in koji. But maybe someone more experienced with mock could finger point
> > the advantages of mock compare to containers ?
> >
> > I like koji but our devs hate the time it takes to build RPMs. I have to
> > say half of the time is taken by mock deploying the buildroot and our
> > builders don't have the luxury to be on SSDs ;)
> >
> > Thomas
> >
> >
> >
> > --
> > buildsys mailing list
> > buildsys(a)lists.fedoraproject.org
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> >
>
>
>
>  --
>  Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
>  contact me at public(a)[mysurname].eu
>
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
>
>
>
>  --
>  Regards,
>  Bohuslav "Slavek" Kabrda.
>
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>  


-- 
Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
contact me at public(a)[mysurname].eu

[Jay Greguske]

Just adding a +1 here. I would be very interested in this work too.

- Jay


On 03/20/2014 07:19 AM, Alan Franzoni wrote:
>  Hello Thomas,
>  our experience with koji showed that we used very little of its
>  functionality but we took a large performance hit from using it; hence
>  we first started using mock and a manually built repository creation
>  system, now we're replacing mock with this docker-rpm-builder.
>  
>  I'll do some benchmarking for mock-vs-drb later on.
>  
>  bye,
>  
>  
>  
>  On Mon, Mar 17, 2014 at 2:57 AM, Thomas Guthmann <tguthmann(a)iseek.com.au
>  <mailto:[e-mail]>> wrote:
>  
>      Hi,
>  
>          https://github.com/alanfranz/__docker-rpm-builder
>          <https://github.com/alanfranz/docker-rpm-builder>
>  
>      I had a thought about it but never found the time to look more into
>      koi internals to see how we can hook it.
>  
>      It would be beneficial in time to replace mock by docker (or plain
>      LXC) in koji. But maybe someone more experienced with mock could
>      finger point the advantages of mock compare to containers ?
>  
>      I like koji but our devs hate the time it takes to build RPMs. I
>      have to say half of the time is taken by mock deploying the
>      buildroot and our builders don't have the luxury to be on SSDs ;)
>  
>      Thomas
>  
>  
>  
>      --
>      buildsys mailing list
>      buildsys(a)lists.fedoraproject.__org
>      <mailto:[e-mail]>
>      https://admin.fedoraproject.__org/mailman/listinfo/buildsys
>      <https://admin.fedoraproject.org/mailman/listinfo/buildsys>
>  
>  
>  
>  
>  -- 
>  Latest blog post: Deploying WAR on Heroku http://t.co/M4krW2Mu
>  contact me at public(a)[mysurname].eu
>  
>  
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>