Drawing lessons from fatal SELinux bug #1054350

[Adam Williamson]

On Tue, 2014-01-28 at 10:39 +0800, Mathieu Bridon wrote:
> On Mon, 2014-01-27 at 15:11 -0700, Kevin Fenzi wrote:
> > On Mon, 27 Jan 2014 10:18:56 -0500
> > Matthew Miller <mattdm at fedoraproject.org> wrote:
> > 
> > ....snip....
> > 
> > > * possibly adding a "what should users test?" field to the update
> > > info.
> > > 
> > >   I know that there's a "notes" field in the update, but maybe it'd
> > > help to explicitly include testing instructions?
> > > 
> > >   Each package in the pkgdb (or in git, or wherever) could have
> > >   a standard list included in each update as the default (for
> > > example, for 'calc', it might be to try `calc -q
> > > read /usr/share/calc/regress.cal`. That would duplicate a likely
> > > smoke-test, though, so maybe also "run interactively and make sure
> > > basic math works".
> >  
> > >   Then, each update could also optionally (and this would be
> > > presented in bold if it were used) say something like "New release
> > > adds log() function; please test that it works", or "Severe bug where
> > > 1+1=3 corrected; please test that the answer now corresponds with
> > > consensus reality."
> > 
> > I could have sworn we already had something like this where bodhi would
> > add a link to a wiki page for test plan on a package if that wiki page
> > existed. I can't seem to find it now, so perhaps it was just something
> > we talked about, but never implemented. 
> 
> Nope, you're right. :)
> 
> https://github.com/fedora-infra/bodhi/blob/develop/bodhi/model.py#L191
> 
> For example, the test case pages for the package 'foo' need to be added
> to the 'Category:Package foo test cases' category.
> 
> There's also an option in the config file which must be switched on for
> this to work: 'query_wiki_test_cases'
> 
> And here is an update where this is, in fact, actually used:
>     https://admin.fedoraproject.org/updates/FEDORA-2014-1465/

This is the package test plan thing - the QA docs on it are at
https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation . I
usually use xorg-x11-drv-nouveau updates as a handy example of it in
action. You can create a test case for a package and add it to the
category Category:Package_(packagename)_test_cases (where 'packagename'
is the .src.rpm name), and it will show up in Bodhi like this.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net