The QA Problem

[Dave Jones]

On Wed, Jan 03, 2007 at 02:41:36PM -0500, Will Woods wrote:

 > Task #1: Testing updates to stable releases
 > Task #2: Testing new releases before they go out (rawhide, TestX, etc.)
 > Task #3: Bug triage (this is the stuff we miss in #1 and #2)
 > Task #4: Writing tools and docs to make the previous tasks easier
 > 
 > Examining the first task - we don't currently have the *dedicated*
 > manpower to *guarantee* that every package will be tested and approved
 > by the QA group before it leaves updates-testing and goes into updates.
 > 
 > I don't think we actually lack for *available* manpower. AFAIK there's
 > plenty of people willing and able to install packages from
 > updates-testing. 

We could definitly use more.  I've been bitten a few times by silly
bugs that should have been caught by users running -testing,
which didn't become obvious until it got moved to -final.

One potential way to increase usage of -testing is to offer
a question in firstboot whether or not to enable it.
As Luis mentioned, he didn't even know about this until recently.
Makes me wonder just how many other users we have who aren't
aware of it.

Quite often I see in bugzilla.. 'This should be fixed in the
build in updates-testing, can you give it a try?' and then
the bug goes silent.  Eventually after it gets pushed to final
the reporter comes back "Yes, that fixed it".  We're definitly
missing out on some potential testers.

		Dave

-- 
http://www.codemonkey.org.uk