On Wed, Mar 5, 2014 at 6:58 AM, Sandro red Mathys <red@fedoraproject.org> wrote:<br>
<blockquote type="cite"><div class="plaintext" style="white-space: pre-wrap;">Heads-up: I've taken ownership for the external need "Automatic
Smoketests on Image Build" [0]
Testing an image takes time and resources, and with several images, it
takes several times that. Since that simply doesn't scale - well,
doesn't even work well without scaling - we want to automate the
smoketests using Taskotron. </div></blockquote><div><br></div><div>rpm-ostree has included in it an (unnamed) custom qemu-based testing hypervisor, which derives from gnome-continuous. Most of the code is in here:</div><div><a href="https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/testbase.js">https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/testbase.js</a></div><div><br></div><div>What makes the rpm-ostree testing hypervisor fundamentally different from a lot of other testing systems out there is it works without IP networking.</div><div><br></div><div>Basically:</div><div><br></div><div>1) Use libguestfs to inject testing code into the guest</div><div>2) The host offers a dbus-over-virtio-serial channel for guests to request things like screenshots</div><div>3) Watch the systemd journal from the guest for events</div><div><br></div><div>Again, operating without guest IP networking and having the ability to watch early boot events like the console.log and systemd journal are key.</div><div><br></div><div>Where I'm going with this eventually is that this testing code acts as a basic smoketest, performed on the same physical host as the image (tree) composition servers. If that passes, then we have an OS version that can be sanely provisioned at a larger scale - say to OpenStack, or to things like AutoTest/Beaker physical hardware inventory/provisioning systems.</div><div><br></div><div>For example the basic smoketest:</div><div><a href="https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/task-smoketest.js">https://github.com/cgwalters/rpm-ostree/blob/master/src/autobuilder/js/tasks/task-smoketest.js</a></div><div><br></div><div>All it does is watch the systemd journal for two events. If we reached multi-user.target, we're good. If any service crashed, we failed.</div><div><br></div><div>This testing code is rather deeply intertwined with the larger rpm-ostree/ostree deployment model - but I'd consider patches to help test traditional RPM-on-the-client OSes.</div><div><br></div><div>(Really it's not even *that* much code, it wouldn't be too hard to fork/rewrite it)</div><div><br></div>