Installed tests

[Colin Walters]

On Tue, 2013-05-28 at 20:13 -0400, Matthias Clasen wrote:
> Hi,
> 
> in upstream GNOME, we're starting to convert the 'make check' style
> tests in many modules into installed tests 

The most important URL is this one:

https://live.gnome.org/GnomeGoals/InstalledTests

The most recent status update is here:
https://mail.gnome.org/archives/desktop-devel-list/2013-May/msg00014.html

But at a very high level, there are two ways in which the InstalledTests
can be consumed by Fedora:

1) Human testers can perform manual reverse dependency testing
   Like I said on the wiki page, the GLib test suite covers a *lot*
   of lower level stuff (namely glibc and kernel) that we expect to
   work.  It's possible for a human to update the kernel, and
   "yum install glib2-tests", and run them to ensure they work.
2) As part of automated testing.  The most important thing to understand
   here is that "Type=session" tests are most effectively run under
   an autologged-in VM.  But it'd be mostly possible to run the current
   GNOME installed test corpus in a mock container with Xvfb +
   dbus-launch, but at least e.g. the clutter tests would have to be
   skipped in such an environment.  We could probably teach clutter
   how to check for GLX before running its tests.

I'd love to see InstalledTests propagate outside of GNOME of course.  If
anyone tries to do that, please let me know how it works out!  For
example, I suspect most projects don't need Type=session and maybe we
should define a Type=headless.