Ideas for analyzing the history of blocker bugs

[Adam Williamson]

One of the items on the Fedora 17 QA retrospective -
https://fedoraproject.org/wiki/Fedora_17_QA_Retrospective - is a
suggestion from Bruno that we could perhaps gain some useful insights by
analyzing the (by now considerable) corpus of blocker bugs from previous
releases, as a way perhaps to identify likely areas of focus for future
development and testing. Bruno has promised that he'll post some more
specific ideas soon, but I wanted to kickstart a thread on the topic in
case anyone else has some. Here's a few of my thoughts to get the ball
rolling...

The most obvious area, perhaps, would be to look at the components
against which the most blockers are filed. That's so easy to do it may
be worth doing anyway, but I suspect the result will be quite
predictable and something we're all more or less aware of anyway: I
would expect the majority of blocker bugs to be in anaconda, then in the
other obvious early-boot critical components (kernel, plymouth, systemd,
udev etc), firstboot, preupgrade, and image generation stuff like
livecd-tools. So I'm not sure that would tell us much we don't already
know, but we might be surprised.

One area that may be more interesting, I guess, would be to look at
various timing issues. One key one would be 'how long it takes for bugs
to be a) nominated and b) accepted as blockers, after they are
reported'. I've come across a few cases before where the answer seemed
to be 'too long' - it would be good to know if they were outliers, or if
we have a consistent issue with not identifying quickly enough that bugs
are blockers. Of course, we could look at the amount of time it takes to
progress through all the other steps of the blocker process too.

So, that's my idea, anyhow :) Do others have thoughts on what kind of
analysis might be interesting/useful? Bruno, can you contribute your
thoughts when ready? Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net