proposal: 'base' validation matrix

[Adam Williamson]

Hey, folks. So, right now, as most of you know, we have two release
validation 'matrices' - sets of tests we run and document on each
candidate build of Fedora to check compliance with the release criteria.

A few of us have been working in ticket #151 -
https://fedorahosted.org/fedora-qa/ticket/151 - to ensure the tests are
in line with the criteria, particularly that we have tests to cover each
criterion, and we've introduced a few new tests to cover criteria that
weren't previously covered.

Some of these don't really fit easily into the category of
'installation' tests or 'desktop' tests, so I'm proposing we introduce a
new, third matrix, which I'm currently thinking of as 'base'. There have
already been three tests written in this proposed new category:

https://fedoraproject.org/wiki/QA:Testcase_base_firstboot
https://fedoraproject.org/wiki/QA:Testcase_base_startup
https://fedoraproject.org/wiki/QA:Testcase_base_system_logging

there will likely be more as we go through the Beta and Final criteria.
I've thrown up a quick draft of how the matrix might look here:

https://fedoraproject.org/wiki/User:Adamwill/Draft_base_validation_matrix

Just wanted to make sure the group knows about this week, and ask for
feedback on the test cases, the matrix and the overall plan here - do
you agree a third matrix is the best way to organize things? Any ideas
for naming, etc? Any comments on the test cases written so far? Thanks!
(also thanks to Athmane Madjouj and Rui He for helping out with this
work.)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net