Packages with missing %check

[Richard W.M. Jones]

On Thu, Feb 27, 2014 at 12:12:59AM -0800, Adam Williamson wrote:
> On Thu, 2014-02-27 at 08:09 +0000, Richard W.M. Jones wrote:
> > On Wed, Feb 26, 2014 at 06:50:56PM -0800, Adam Williamson wrote:
> > > On Wed, 2014-02-26 at 21:53 +0000, Richard W.M. Jones wrote:
> > > >   It happens too often in
> > > > Rawhide, and a simple test (in %check or elsewhere) could fix it.
> > > 
> > > You can't really test a system boot in a package's %check. That's very
> > > definitely not what it's for.
> > 
> > It turns out you can.  Read the link I posted:
> > 
> > http://people.redhat.com/~rjones/qemu-sanity-check/
> 
> I've read it. I'm not exactly convinced.

If there is a specific problem, please mention what it is.  Otherwise
there is no action I can take.

> > Yes, it doesn't catch every possible problem, but the kernel is
> > sometimes pushed out with major problems that could be caught with
> > simple testing.
> 
> IIRC you had one example for this, and the bug turned out to be racy and
> hardware-dependent, which is why the kernel devs didn't see it in the
> tests they *do* actually do.

*One*?  There's loads of them.  Regularly the kernel doesn't boot on
qemu, most recently just two days ago (although that "fixed itself"
before I managed to get around to filing a bug).

JUST the bugs in the kernel & qemu filed or fixed in 2013/2014:

https://bugzilla.redhat.com/show_bug.cgi?id=872831
https://bugzilla.redhat.com/show_bug.cgi?id=1033971
https://bugzilla.redhat.com/show_bug.cgi?id=1034433
https://bugzilla.redhat.com/show_bug.cgi?id=998722
https://bugzilla.redhat.com/show_bug.cgi?id=991808
https://bugzilla.redhat.com/show_bug.cgi?id=857026
https://bugzilla.redhat.com/show_bug.cgi?id=962079
https://bugzilla.redhat.com/show_bug.cgi?id=863978

non-x86:
https://bugzilla.redhat.com/show_bug.cgi?id=1066581
https://bugzilla.redhat.com/show_bug.cgi?id=1036742

non-Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=998065

In one sense none of this is surprising at all.  The kernel
(especially) and qemu are pushed out with next to no testing.  They
are huge complex pieces of software.  The fact they work some of the
time is a miracle really.

And none of this includes the frequent unreproducible heisenbugs that
I see.  I only file bugs about things that are reproducible for
obvious reasons, but if you run the libguestfs test suite you'll see
that it often fails because of problems that are obviously in the
kernel but can't be reproduced when run a second time.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org