Draft Fedora 21 Test Plan

[Adam Williamson]

Hi, folks! So, I quickly bashed out that draft F21 Test Plan I've been
threatening to write for the last month or so.

https://fedoraproject.org/wiki/User:Adamwill/Draft_Fedora_21_test_plan

So, what's the idea here?

Really, I'm just trying to come up with a starting point for all the
decisions we need to make and planning we need to do for Fedora 21
testing, especially release validation testing, especially as regards
Fedora.next. There's some mention of other stuff in there, but what I'm
really trying to think about is what we need to do to extend our test
processes to cover all the stuff that's coming as a part of .next.

I think the most interesting points are these:

* There are several practical implications from the test plan - just
Work We Need To Do. Most obviously, we need to draw up release criteria
and supporting test cases for the Fedora.next Products. We also will
need to adjust the
https://fedoraproject.org/wiki/QA/SOP_Release_Validation_Test_Event SOP
to include those cases/matrices as they're created, and adjust the
https://fedoraproject.org/wiki/Release_criteria page to refer to/include
the Product-specific release criteria as *they're* created.

* Responsibilities! Particularly, in this *draft* Test Plan, I've
suggested that creating the Product-specific criteria and test cases
should be the responsibility of the relevant Working Groups, with
assistance from the QA team. They would also be jointly responsible,
along with the QA team, for conducting Product-specific testing, on the
general principle that the more Product-specific a bit of testing is
(and the more domain-specific expertise it needs), the more it's the
WG's responsibility and the less it's QA's responsibility. Note that
(again, as a suggestion in this draft) the QA team is still responsible
for Fedora 21 testing *overall* - i.e. it's the QA team's responsibility
to make sure the WGs do the stuff assigned to them in the plan. If that
makes sense. Discussion welcome!

* Feasibility. This is, of course, one I really want to nail down. It
will require, though, that the Product-specific release criteria and
test cases / matrices get written. Once we have those, we can try and
make a realistic judgement as to whether we as a project - the QA team
and the WGs - actually have the resources to complete the minimal
necessary level of testing within the scope of the F21 schedule. If we
don't think that's going to be the case, we can take that to FESCo, and
we can decide what to do about it that way. In the meantime I've
provided a *very* vague 'Required resources' section just to sort of
flag this up in a modest way.

I'd really, really appreciate feedback on this draft from all interested
parties - QA folks, WG folks, FESCo, anyone at all reading this who has
an opinion. Please, bikeshed away. It all helps.

I'll aim to kick off discussion between the WGs and QA about creating
Product-specific release criteria and test cases next week. Obviously
that's *somewhat* subject to feedback on this very draft, but I can't
really envisage a world in which we aren't going to need those two
things to happen, and whoever's job we ultimately decide it is, I don't
*think* that me just kicking off an attempt to draft them can be a bad
thing.

Thanks folks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net