Related to our disposable clients initiative in Taskotron, I'm working on ways how to boot and manage those clients. In order to know how to boot them, I need to know how they are created (e.g. cloud images require special magic with cloud-init). That's why I spent some time in the past looking at different ways how to either create custom images or re-use existing ones. The simplest approach turned out to be (in my view) a tool called virt-builder. It's a part of libguestfs project and it's a super easy way to create Fedora disk images - they have them already pre-created and stored on the server, so it just downloads them and customize them a little bit per your requirements:
$ virt-builder fedora-21 --root-password password:fedora --install vim-enhanced --update
I decided to try to use it for creating the base image for our disposable clients. The idea is that virt-builder is run once a day on a server and it generates a fresh new VM disk image. We then use this disk image for all our disposable clients spawned in the future, until a new disk image is generated the next day. Rinse and repeat.
The benefit of this approach (over e.g. official cloud images which might be updated once per month or twice per year) is: * the clients are always up-to-date * it's very light on bandwidth - instead of downloading updates for every client on every task run, it's done once per day, centrally * it's much faster - the clients just boot and execute the task, they don't spend time upgrading each boot * it's less error-prone - there is a never-ending story of failures when you communicate over network thousands times per day. We know it really well. By updating the client just once, the chance for a network error is much lower, and we can easily repeat it if it fails. Also, it's not a big deal if it completely fails some day, our tasks continue to be executed without a problem, just with a day-old packages installed. * can be integrated with fedmsg "updates pushed" notification to be almost instantly in sync with the latest updates * it helps with rescheduling tasks and reproducing issues - if we keep last X-number of VM images, we can easily replay past tasks with exactly the same package set as it had during that time
Subsequently I realized that we will probably also want to have several different base disk images, not just one. For example, certain GNOME checks might require most of GNOME installed, which would mean downloading a lot of packages every time they are run. But with virt-install, we can pre-create different base images for different purposes, and then let the task define which image to use. So for GNOME checks, we would prepare a base image with full GNOME installed.
To make all of this manageable, I created a wrapper around virt-builder and called it taskotron-vmbuilder (ideas for a better name are welcome). The code is available here:
https://bitbucket.org/fedoraqa/taskotron-vmbuilder
The concept is that you can define templates and then run vmbuilder which takes care of the rest - it uses virt-builder to create the image, it makes sure that libtaskotron is installed, it configures stuff like root password and ssh key. We can add any other functionality that is needed. The simplest way to try this is this:
$ ./vmbuilder.py templates/f21-minimal.yaml
There are sample templates for minimal installation and Server and Workstation products. For more details, look into READMEs, it's fairly well documented, I think. My idea is that we will run this daily on the server, store the resulting images somewhere, and update the pointer to the "current one".
I still haven't created project and repo in Phab, but if you approve this as a way forward, I will.
Thanks, Kamil