P2P Packaging/Koji Cloud

[seth vidal]

On Wed, 7 Dec 2011 18:31:27 +0100
Denis Arnaud <denis.arnaud_fedora at m4x.org> wrote:

> 2011/12/7 seth vidal <skvidal at fedoraproject.org>
> 
> > I've looked into spawning virt instances to do building and it is
> > pretty doable. The problem with them being offered by volunteers is
> > trust [...]
> >
> 
> You are right. I had not thought at that... how naive of me :(
> 
> The volunteers/trustees would sign the builds with their own private
> keys, for instance with their FAS keys. Then, we could have some
> "trustworthiness" ratings for all the submitters, like we have today
> for the packagers (new packager, proven-packager, sponsor). While the
> submitter is still not trusted, the centralised Koji infrastructure
> can duplicate the build, and check that it gives the same results.
> And even when the submitter is trusted, some random duplicate builds
> can occur. If the submitter taints the builds, it will be flagged as
> a potential "fraud". A human being would have to have a look at it
> then.
> 


Well it's beyond that - we still can't trust the build host isn't
compromised in such a way to also add trojans to a mock chroot we then
build inside of. In short, if people are volunteering VMs for us - we
have to be incredibly suspicious of those VMs and from what they are
built.

If I were going to use random vm's I'd want to:
1. connect using ssh
2. push over my own rpm/python/etc binaries
3. checksum all the rest of the installed (and running) software
4. verify those checksums versus my known good set
5. THEN push over the pkgs to be built

If we run this on ec2 or cloudservers then we can build up our own
image and use that for the initial "trusted" environment. ec2 has the
benefit of letting you share an image with others and from there we can
build.

my goal is a very short stack of code such that anyone can run it to
build pkgs - if only b/c they don't want to run mock locally b/c of
limited local bandwidth or cpu time.

> Or, the VMs could do "scratch" builds (only). When those builds are
> successful, the VMs then just act as a standard clients to the
> central Koji servers, and the packages are re-built in that safe
> infrastructure. Overall, the central Koji infrastructure would be
> off-loaded from all the scratch builds, as well as from the failed
> builds. Which is already not so bad, is it?


I think a reasonable goal is:
 - make it trivial to build pkgs in a remote cloud instance from
 arbitrary yum repository sources
 - implement the requirements to make this trustworthy enough for koji
   to do it, too.

> 
> 
> That would be very cool. Do you intend to use DeltaCloud (
> http://deltacloud.apache.org/), or something like that?

I'm using libcloud, actually. I'm interested in pursuing this in
python, not ruby.

> >
> 
> Yes, that is fully in-line, and very interesting!

 
> PS: why isn't there a virtualisation SIG? As there is already a
> mailing list, it may be just a question of adding the corresponding
> Wiki page?

I thought there was a cloud sig or something. To be honest the sigs are
mind-numbingly confusing to me as to 1. what they do and 2. which ones
actually still exist/do something. So I tend to not pay much attention
to them.

-sv