P2P Packaging/Koji Cloud
rmarko at redhat.com
Wed Dec 7 15:01:06 UTC 2011
On 12/07/2011 02:46 PM, Denis Arnaud wrote:
> RedHat-hosted Koji servers offer an invaluable service by allowing all
> of us, package maintainers, to build all of "our" Fedora packages. I
> guess that that infrastructure is not cost-less for RedHat and and the
> quality of service is great (for instance, the wait in the queues,
> before Koji actually builds the packages submitted via the
> command-line client, is not so long).
> As Fedora is pretty advanced in the cloud/virtualisation arena, we
> could imagine a "Koji Cloud", hosted on VMs offered by volunteers. For
> instance, I could contribute a few VMs in Europe (hosted on
> http://www.ovh.co.uk/). Our Cloud SIG
> (https://fedoraproject.org/wiki/Cloud_SIG) and/or Virt ML
> (https://admin.fedoraproject.org/mailman/listinfo/virt and
> ET (http://et.redhat.com/) colleagues could help designing and
> implementing the following infrastructure:
> * VM template/images, ready to be started on the volunteer's servers
> everywhere in the world, 24x7.
> - SSH public keys of Koji administrators would be part of the
> images, so that they can have an easy access to them, just in case.
> - Those VMs would update themselves automatically.
> - The containers could be standardised as well. For instance,
> ProxMox/OpenVZ or Fedora/CentOS with libvirt.
> * A directory (LDAP, or something less centralised, like the address
> book of Skype, for instance), keeping track of all those VMs:
> - with the corresponding last known status;
> - with the VM configurations (Fedora/CentOS release, CPU, memory,
> disk usage, etc);
> - with some rating corresponding to their quality of service
> (build duration, reliability of the VM, MTBF, etc).
> * A dispatcher system:
> - which would route the Koji build requests to available VMs;
> - collect the outcome of the builds (logs, RPM packages,
> statistics, QoS, etc) and store them in the current ("centralised")
> Koji infrastructure.
> As I am not a specialist of all those technologies, I may have
> forgotten a lot of things, but you get the idea.
> Doesn't it sound great? Does it sound realisable? Am I crazy to dream
> to such an infrastructure?
I'm currently writing a proposal of similar architecture for testing
purposes. Looks like the core -- community provided virtual machines is
the common component for all this stuff so if designed correctly it can
be shared for testing/koji/whatever.
I will let you know when the proposal is done so we can discuss the
More information about the devel