proposal: 'base' validation matrix

[James Laska]

On Thu, 2011-07-07 at 16:26 -0700, Adam Williamson wrote:
> On Thu, 2011-06-30 at 10:09 -0700, Adam Williamson wrote:
> 
> > > > Any ideas
> > > > for naming, etc? Any comments on the test cases written so far? Thanks!
> > > 
> > > One thought, the 'base' matrix page looks a bit sparse.  
> > 
> > This is intentional, I really just wanted to create the table itself
> > right now, not the complete finished page.
> 
> I've now expanded the proposal into a full template:
> 
> http://fedoraproject.org/wiki/User:Adamwill/Draft_base_validation_matrix
> 
> further comments welcome! There will probably be a few more tests once
> we're done checking Beta and Final criteria for concordance with the
> criteria.

Looks good.  Do we need to run these tests on all primary architectures
like we do for the installer?  Meaning, is it possible/likely that the
outcomes will differ across those arches?

> > > I was about to
> > > suggest that we add a standard header to the new 'base' matrix with some
> > > basic instructions, but I wonder if something else might be more
> > > appropriate.  When Andre and Hurry announce test runs, instead of
> > > linking to each individual test matrix, we'd send a link to the
> > > appropriate Category:Test_Results page (e.g.
> > > https://fedoraproject.org/wiki/Category:Fedora_15_Final_RC_Test_Results).  
> > > 
> > > What if we move the stock "instructions" (see
> > >
> > https://fedoraproject.org/wiki/Test_Results:Fedora_15_Final_RC1_Install#Instructions) out of each individual page, and into the category page?  Or at the very least, make it an trancludable [1] wiki template {{Fedora_15_Test_Results_Instructions}}.
> > 
> > I'd be opposed to this, for a couple of reasons: the instructions may
> > differ between matrices (I think they do, for desktop and install) and I
> > like the simplicity of having a single page you can point people to for
> > one particular matrix. Using a template is probably feasible for any
> > bits that are duplicated between pages, I'll take a look at that.
> 
> Writing the full proposed page reinforced my view on this: as you'll
> see, the instructions are different from those in the desktop and
> install matrices, and the proposed security matrix. I don't think we
> could reconcile the instructions between the various matrices, I think
> they do need to differ in various particulars, and maintaining a
> template to keep the common elements together while allowing for
> substantial variation would be quite difficult and hard to read.

Yeah, could be.  We'll find out with the criteria templates :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20110708/ac8ce167/attachment.bin