Desktop validation testing expansion

[Adam Williamson]

Hi, everyone.

I'm sending this email to more widely announce something we've been
working on for a while. I've spoken to leaders of the XFCE, KDE and LXDE
teams about this. We'd like to propose expanding the desktop validation
testing that was initiated in the Fedora 13 cycle.

During F13, these tests were used to check for blocker bugs only for the
GNOME desktop. We did run the tests against some of the other desktops
at some checkpoints, but failures from those runs weren't considered as
blocking the release.

For the Fedora 14 cycle, I'd like to try to run the desktop validation
test set - that's the testing documented at
https://fedoraproject.org/wiki/QA:Desktop_validation_testing - on all
four major desktops at each checkpoint, and consider failures at the
relevant level to be release blocking bugs.

We've made some adjustments to the test suite already to reflect the
input from each SIG, and we may yet work on refining the procedure a
little more, but the outline of the plan is simple enough.

This would be technically on an experimental basis, as the GNOME testing
was during the F13 cycle; if it seems to be causing too much trouble and
making it impractical to work to our release plans, we might reconsider
the idea during the cycle. But we didn't have to do that for F13, and
I'd hope we won't have to for F14 either.

The leaders of the various desktop SIGs are fairly confident that their
SIGs will be able to contribute in running the tests and fixing blocker
issues promptly as they're identified. I think the QA group will also be
able to contribute to this process, particularly in carrying out the
testing.

If anyone from QA or any of the SIGs has comments or concerns on the
proposal, please air them now! Thanks a lot :)

If we do end up going ahead with the idea, I'll follow up with all of
the SIGs when we hit the first round of validation testing to keep you
all informed on how the process works and how to contribute to it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net