handling lots of builds on the same copr and same distro/chroots

[Seth Vidal]

So I'm trying to figure out how to sensibly handle the following 
situation:

user creates a new copr with fedora-16-i386 and fedora-16-x86_64

submits 3 pkgs to build: - that's buildid 45
submits 4 more pkgs to build: that's buildid 46
submits 2 more pkgs to build: that's buildid 47

If they are all building at the same time - then we're going to have a 
conflict when createrepo will try to run (to say nothing of rsync for 
syncing pkgs over).

I was thinking of tracking per-copr builds and locking the build process 
such that if the copr has a build active another one cannot be added to 
the queue until the earlier one has finished.

That's going to involve moving some code around so I can track which 
builds are active a bit more easily.

Anyone have any better ideas?

Thanks,
-sv

[Bohuslav Kabrda]

----- Original Message -----
>  So I'm trying to figure out how to sensibly handle the following
>  situation:
>  
>  user creates a new copr with fedora-16-i386 and fedora-16-x86_64
>  
>  submits 3 pkgs to build: - that's buildid 45
>  submits 4 more pkgs to build: that's buildid 46
>  submits 2 more pkgs to build: that's buildid 47
>  
>  If they are all building at the same time - then we're going to have
>  a
>  conflict when createrepo will try to run (to say nothing of rsync for
>  syncing pkgs over).
>  
>  I was thinking of tracking per-copr builds and locking the build
>  process
>  such that if the copr has a build active another one cannot be added
>  to
>  the queue until the earlier one has finished.
>  
>  That's going to involve moving some code around so I can track which
>  builds are active a bit more easily.
>  
>  Anyone have any better ideas?
>  
>  Thanks,
>  -sv
>   
I totally agree with the idea of "one build per copr at a time".
Another reason for builds to wait is that we don't know whether they depend on results
of each other, so we shouldn't run them in parallel. But one by one.

-- 
Regards,
Bohuslav "Slavek" Kabrda.

[Adam Miller]

On Wed, Jan 16, 2013 at 01:58:10AM -0500, Bohuslav Kabrda wrote:
>  ----- Original Message -----
>  > So I'm trying to figure out how to sensibly handle the following
>  > situation:
>  > 
>  > user creates a new copr with fedora-16-i386 and fedora-16-x86_64
>  > 
>  > submits 3 pkgs to build: - that's buildid 45
>  > submits 4 more pkgs to build: that's buildid 46
>  > submits 2 more pkgs to build: that's buildid 47
>  > 
>  > If they are all building at the same time - then we're going to have
>  > a
>  > conflict when createrepo will try to run (to say nothing of rsync for
>  > syncing pkgs over).
>  > 
>  > I was thinking of tracking per-copr builds and locking the build
>  > process
>  > such that if the copr has a build active another one cannot be added
>  > to
>  > the queue until the earlier one has finished.
>  > 
>  > That's going to involve moving some code around so I can track which
>  > builds are active a bit more easily.
>  > 
>  > Anyone have any better ideas?
>  > 
>  > Thanks,
>  > -sv
>  > 
>  
>  I totally agree with the idea of "one build per copr at a time".
>  Another reason for builds to wait is that we don't know whether they depend on
> results of each other, so we shouldn't run them in parallel. But one by one.
>  
>  -- 
>  Regards,
>  Bohuslav "Slavek" Kabrda. 
+1

I agree that it makes sense to limit it to "one build per copr at a
time".

-AdamM