problems with QA:Testcase_base_startup

[Adam Williamson]

On Thu, 2014-09-18 at 08:31 -0600, Mike Ruckman wrote:
> On Thu, Sep 18, 2014 at 06:42:07AM -0400, Kamil Paral wrote:
> > In the base installation matrix
> > https://fedoraproject.org/wiki/Test_Results:Fedora_21_Alpha_RC1_Base#Test_Matrix
> > 
> > we have this test case:
> > https://fedoraproject.org/wiki/QA:Testcase_base_startup
> > 
> > It it is marked as "Alpha" in the matrix. However, it depends on disk encryption, which is a Beta requirement. That is a problem, if disk encryption doesn't work, you can't test the rest of the test case. That means you can test Alpha criteria because you are stuck on a broken Beta feature.
> > 
> > Furthermore, the test case basically duplicates https://fedoraproject.org/wiki/QA:Testcase_partitioning_guided_encrypted , from my POV.
> > 
> > So I suggest to amend Testcase_base_startup and change this sentence:
> > "Follow all defaults, with the exception that you should encrypt a system partition (e.g. the root partition)."
> > to this:
> > "Follow all defaults, with the exception that you may encrypt a system partition (e.g. the root partition)."
> > 
> > And of course the expected results would be a slightly differently formulated as well.
> > 
> > This will fix the discrepancy (Alpha test case depending on Beta features), and remove the duplication.
> > Thoughts?
> 
> I agree with John. Yanking the encryption bit completely makes good sense.

So, I think something's gotten lost somewhere along the way, here. My
recollection is that Base is supposed to test keymaps, but it doesn't
actually seem to, so indeed it seems to be a dupe of the encryption test
case so far as encryption goes.

I agree this nexus seems rather messy. What we need to test, in broad
terms, is:

If you install and boot either graphically or non-graphically with
encryption and a non-US keymap, everything should work.

The Base test case in particular does read a bit like a hot mess with
its attempt to cover both graphical and non-graphical cases. Perhaps it
would be best to split it in two.

It looks like at present
https://fedoraproject.org/wiki/QA:Testcase_desktop_login covers keyboard
layouts, but not base_startup.

I'm not really sure what the best way to express all this is. Any
thoughts?

For the short term, I'd be +1 to ditching the encryption bits of the
base_startup case as they're simply duplicating the guided_encryption
case. We should also stop claiming it's associated with the keyboard
layout criterion if it isn't. But we definitely *do* need to be testing
non-US layout encryption.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net