Automatic Smoketests for the Cloud Images: What to Test?

Colin Walters walters at verbum.org
Wed Mar 5 22:51:58 UTC 2014


On Wed, Mar 5, 2014 at 6:58 AM, Sandro red Mathys 
<red at fedoraproject.org> wrote:
> Heads-up: I've taken ownership for the external need "Automatic
> Smoketests on Image Build" [0]
> 
> Testing an image takes time and resources, and with several images, it
> takes several times that. Since that simply doesn't scale - well,
> doesn't even work well without scaling - we want to automate the
> smoketests using Taskotron. 
> 

rpm-ostree has included in it an (unnamed) custom qemu-based testing 
hypervisor, which derives from gnome-continuous.  Most of the code is 
in here:
https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/testbase.js

What makes the rpm-ostree testing hypervisor fundamentally different 
from a lot of other testing systems out there is it works without IP 
networking.

Basically:

1) Use libguestfs to inject testing code into the guest
2) The host offers a dbus-over-virtio-serial channel for guests to 
request things like screenshots
3) Watch the systemd journal from the guest for events

Again, operating without guest IP networking and having the ability to 
watch early boot events like the console.log and systemd journal are 
key.

Where I'm going with this eventually is that this testing code acts as 
a basic smoketest, performed on the same physical host as the image 
(tree) composition servers.  If that passes, then we have an OS version 
that can be sanely provisioned at a larger scale - say to OpenStack, or 
to things like AutoTest/Beaker physical hardware inventory/provisioning 
systems.

For example the basic smoketest:
https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/task-smoketest.js

All it does is watch the systemd journal for two events.  If we reached 
multi-user.target, we're good.  If any service crashed, we failed.

This testing code is rather deeply intertwined with the larger 
rpm-ostree/ostree deployment model - but I'd consider patches to help 
test traditional RPM-on-the-client OSes.

(Really it's not even *that* much code, it wouldn't be too hard to 
fork/rewrite it)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/cloud/attachments/20140305/d380bcd7/attachment.html>


More information about the cloud mailing list