Re: Updates to disk unmount test case.

[Kamil Paral]

On Tue, Nov 26, 2019 at 9:16 PM Adam Williamson <adamwill(a)fedoraproject.org&gt;
wrote:

>  On Tue, 2019-11-26 at 13:59 -0500, pmkellly(a)frontier.com wrote:
>  >
>  > I assure you that I wasn't being lazy when I created the web draft test
>  > case. It wasn't copied from e'mail, but from my original LibriWriter
>  > file where I composed it. The formatting with the results right under
>  > the test step was a demonstration of a format that has some advantages
>  > over the existing format. Just a little demo; I will change it.
>
>  I think it can be a good format indeed, and I've seen it used in other
>  test contexts, but all our other significant test cases use the format
>  intended by the template where all the test steps are together and all
>  the expected results are together after the test steps. So it'd
>  probably be best to stay consistent with that.
>
>  You can use little writing tricks to help people map each test step to
>  the correct expected result, like you can write in a test step "run
>  command X and then check the output" (not just "run command X"), and
>  then in the results you write something like "The output of command X
>  should be..." I find that's usually enough to make it clear for
>  testers.
>  
And I'm going to heartily disagree with Adam here. If I hate something
about certain our test cases, it's when you have to constantly jump forward
and back and look for expected results somewhere else on the page. This
makes the test case very hard to follow and very easy to unintentionally
omit important actions. Sometimes it makes you repeat the test case just
because you read an important instruction too late. I wanted to rewrite all
of those bad test cases a long time ago, but never forced myself to do it.

The reasonable approach is to write steps one by one in the "How to test"
section, and any time there is an expectation of something immediate,
mention it at the very same step. It is not necessary to include obvious
things. For example, this is not necessary:
1. Log in through virtual terminal. Login should be accepted.
because broken login would obviously prevent you from continuing your test
case steps. So this is enough:
1. Log in through virtual terminal.
But in some cases you want to say something should happen right now that is
not obvious and should be mentioned:
1. Create a disk layout without a swap partition. A warning dialog should
appear, but it should not prevent you from confirming the layout.
or
1. Execute command "foobar". The command should successfully finish (with
return code 0) and its output should contain "all ok".

The "Expected results" section should contain expectations that are
relevant to the final state of the system (although these could be also
written together with the last step in "How to test"), or to a condition
that should be valid throughout the whole testing process. For example:
1. The final installed system must boot successfully and present a login
screen.
or
1. Throughout the whole installation process, all mentions of the system
version or the edition on all screens must identify the current system
correctly (e.g. version is always shown as "31" and edition is always shown
as "Server").