"Testcase base startup" confusion

[Adam Williamson]

On Mon, 2012-12-17 at 06:04 -0500, Kamil Paral wrote:
> This test case is a bit confusing:
> https://fedoraproject.org/wiki/QA:Testcase_base_startup
> 
> As Lukas Brabec found out, it requires people to create an encrypted setup by using the text mode, which is no longer supported by F18 anaconda. We need to adjust it, the question is how.
> 
> I have to say I'm not a fan of complex test cases that require people to do multiple things (first install in graphical mode, then install in text mode, etc etc).
> 
> I propose to:
> a) either split this test case into two separate ones, graphical and text mode, where the graphical one will require disk encryption and the text mode will not
> b) or don't specify graphical/text mode, and don't specify disk encryption, let people choose. Some will do this, some will do that. I don't see why we should require disk encryption when it is already covered by "Anaconda_autopart_(encrypted)_install" test case.
> 
> Thoughts?

The point here is that 'does the install succeed?' and 'does the
installed system boot?' are really separate test cases. Say you install
an encrypted system, right? Say anaconda does everything just right, but
there's a bug in dracut and you can't enter the passphrase on boot of
the installed system.

That's really nothing to do with installation. It's not a bug in the
installer. It shouldn't be reported against anaconda. So it really
shouldn't be part of the 'installation' test case. This was the reason
for splitting this 'startup' test case out on its own in the first place
- to make the distinction between 'failure during install' and 'failure
on boot' clearer. I'm not a fan of 'install' test cases that include
'check the installed system boots'.

I don't mind splitting it out further, at all, but I'm opposed to just
saying 'oh, the install test case covers it', that's going backwards.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net