Re: [fedora-virt] P2P Packaging/Koji Cloud
On 12/07/2011 02:46 PM, Denis Arnaud wrote:
Hello,
RedHat-hosted Koji servers offer an invaluable service by allowing all
of us, package maintainers, to build all of "our" Fedora packages. I
guess that that infrastructure is not cost-less for RedHat and and the
quality of service is great (for instance, the wait in the queues,
before Koji actually builds the packages submitted via the
command-line client, is not so long).
As Fedora is pretty advanced in the cloud/virtualisation arena, we
could imagine a "Koji Cloud", hosted on VMs offered by volunteers. For
instance, I could contribute a few VMs in Europe (hosted on
http://www.ovh.co.uk/). Our Cloud SIG
(https://fedoraproject.org/wiki/Cloud_SIG) and/or Virt ML
(https://admin.fedoraproject.org/mailman/listinfo/virt and
https://fedoraproject.org/wiki/Getting_started_with_virtualization)/RedHat
ET (http://et.redhat.com/) colleagues could help designing and
implementing the following infrastructure:
* VM template/images, ready to be started on the volunteer's servers
everywhere in the world, 24x7.
- SSH public keys of Koji administrators would be part of the
images, so that they can have an easy access to them, just in case.
- Those VMs would update themselves automatically.
- The containers could be standardised as well. For instance,
ProxMox/OpenVZ or Fedora/CentOS with libvirt.
* A directory (LDAP, or something less centralised, like the address
book of Skype, for instance), keeping track of all those VMs:
- with the corresponding last known status;
- with the VM configurations (Fedora/CentOS release, CPU, memory,
disk usage, etc);
- with some rating corresponding to their quality of service
(build duration, reliability of the VM, MTBF, etc).
* A dispatcher system:
- which would route the Koji build requests to available VMs;
- collect the outcome of the builds (logs, RPM packages,
statistics, QoS, etc) and store them in the current ("centralised")
Koji infrastructure.
As I am not a specialist of all those technologies, I may have
forgotten a lot of things, but you get the idea.
Doesn't it sound great? Does it sound realisable? Am I crazy to dream
to such an infrastructure?
Cheers
Denis
Let me start out by saying I like the idea very much. I do see some
challenges.
In a decentralised system, you must take measures to ensure the quality
and reliability of service. Is there any way to verify if a package was
built correctly? Perhaps the VM assigned for building it is accidentally
misconfigured? Could a malicious user compromise one of the VMs used for
building and insert wrong code into one of the packages?
Emanuel