P2P Packaging/Koji Cloud

[Denis Arnaud]

Hello,

RedHat-hosted Koji servers offer an invaluable service by allowing all of
us, package maintainers, to build all of "our" Fedora packages. I guess
that that infrastructure is not cost-less for RedHat and and the quality of
service is great (for instance, the wait in the queues, before Koji
actually builds the packages submitted via the command-line client, is not
so long).

As Fedora is pretty advanced in the cloud/virtualisation arena, we could
imagine a "Koji Cloud", hosted on VMs offered by volunteers. For instance,
I could contribute a few VMs in Europe (hosted on http://www.ovh.co.uk/).
Our Cloud SIG (https://fedoraproject.org/wiki/Cloud_SIG) and/or Virt ML (
https://admin.fedoraproject.org/mailman/listinfo/virt and
https://fedoraproject.org/wiki/Getting_started_with_virtualization)/RedHat
ET (http://et.redhat.com/) colleagues could help designing and implementing
the following infrastructure:
 * VM template/images, ready to be started on the volunteer's servers
everywhere in the world, 24x7.
    - SSH public keys of Koji administrators would be part of the images,
so that they can have an easy access to them, just in case.
    - Those VMs would update themselves automatically.
    - The containers could be standardised as well. For instance,
ProxMox/OpenVZ or Fedora/CentOS with libvirt.
 * A directory (LDAP, or something less centralised, like the address book
of Skype, for instance), keeping track of all those VMs:
    - with the corresponding last known status;
    - with the VM configurations (Fedora/CentOS release, CPU, memory, disk
usage, etc);
    - with some rating corresponding to their quality of service (build
duration, reliability of the VM, MTBF, etc).
 * A dispatcher system:
    - which would route the Koji build requests to available VMs;
    - collect the outcome of the builds (logs, RPM packages, statistics,
QoS, etc) and store them in the current ("centralised") Koji infrastructure.

As I am not a specialist of all those technologies, I may have forgotten a
lot of things, but you get the idea.
Doesn't it sound great? Does it sound realisable? Am I crazy to dream to
such an infrastructure?

Cheers

Denis

[Emanuel Rietveld]

On 12/07/2011 02:46 PM, Denis Arnaud wrote:
> Hello,
>
> RedHat-hosted Koji servers offer an invaluable service by allowing all 
> of us, package maintainers, to build all of "our" Fedora packages. I 
> guess that that infrastructure is not cost-less for RedHat and and the 
> quality of service is great (for instance, the wait in the queues, 
> before Koji actually builds the packages submitted via the 
> command-line client, is not so long).
>
> As Fedora is pretty advanced in the cloud/virtualisation arena, we 
> could imagine a "Koji Cloud", hosted on VMs offered by volunteers. For 
> instance, I could contribute a few VMs in Europe (hosted on 
> http://www.ovh.co.uk/). Our Cloud SIG 
> (https://fedoraproject.org/wiki/Cloud_SIG) and/or Virt ML 
> (https://admin.fedoraproject.org/mailman/listinfo/virt and 
> https://fedoraproject.org/wiki/Getting_started_with_virtualization)/RedHat 
> ET (http://et.redhat.com/) colleagues could help designing and 
> implementing the following infrastructure:
> VM template/images, ready to be started on the volunteer's servers
>
> everywhere in the world, 24x7.
>     - SSH public keys of Koji administrators would be part of the 
> images, so that they can have an easy access to them, just in case.
>     - Those VMs would update themselves automatically.
>     - The containers could be standardised as well. For instance, 
> ProxMox/OpenVZ or Fedora/CentOS with libvirt.
> A directory (LDAP, or something less centralised, like the address
>
> book of Skype, for instance), keeping track of all those VMs:
>     - with the corresponding last known status;
>     - with the VM configurations (Fedora/CentOS release, CPU, memory, 
> disk usage, etc);
>     - with some rating corresponding to their quality of service 
> (build duration, reliability of the VM, MTBF, etc).
> A dispatcher system:
> which would route the Koji build requests to available VMs;
> collect the outcome of the builds (logs, RPM packages,
>
> statistics, QoS, etc) and store them in the current ("centralised") 
> Koji infrastructure.
>
> As I am not a specialist of all those technologies, I may have 
> forgotten a lot of things, but you get the idea.
> Doesn't it sound great? Does it sound realisable? Am I crazy to dream 
> to such an infrastructure?
>
> Cheers
>
> Denis

Let me start out by saying I like the idea very much. I do see some 
challenges.

In a decentralised system, you must take measures to ensure the quality 
and reliability of service. Is there any way to verify if a package was 
built correctly? Perhaps the VM assigned for building it is accidentally 
misconfigured? Could a malicious user compromise one of the VMs used for 
building and insert wrong code into one of the packages?

Emanuel