Hello,
I wanted to tweak a gnome-software upgrade test case [1] lately, and
realized that... I can't. I got lost in the number of templates it's
composed from (try it yourself). Even if you know wiki templates basics, it
was too difficult to even find the page containing the source text that I
saw rendered. And I was afraid that I'll break other pages elsewhere. So I
decided I try to simplify that.
The current situation is that everything gets composed in Testcase upgrade
[2]. It's very complex and uses a lot of parameters. It forwards these
parameters to other templates, and includes them. It's a very clever system
(kudos to Adam), but unfortunately it's very hard to read and edit.
So, I did the following:
1. I replaced anonymous numbered parameters with named ones. So
{{{flavor}}} instead of {{{1}}}.
2. I split dnf steps from gnome-software steps. They are sufficiently
different, merging them together just makes it more complex.
3. I created short separate templates for each test case section
(description, setup, how to test, expected results). Separating it into
chunks makes it more readable and easier to navigate.
4. I introduced "setup" section in testcases, because it allowed to
separate generic steps from specific steps.
5. I moved some of the expected results into the "how to test" section.
That allowed me to make "expected results" section generic, and as a bonus,
the "how to test" section is now much more pleasant to read, because you
don't need to constantly jump to expected results and search whether you
missed something. Things that you should check directly after a particular
step is now written down at that very step. Only generic things to check
not related to any particular step are in "expected results" (that makes
much more sense, imho).
6. I simplified the conditions a bit. The instructions don't need to be
perfectly parametrized, that just makes them hard to edit. So sometimes the
instructions include sentences like "If your disks are encrypted, do X."
even when you're displaying a test case for a non-encrypted upgrade. We're
humans, we can deal with an "If" clause well (we're using this approach
everywhere else), and this makes a lot of stuff much simpler.
7. I removed some old workarounds (F24 ones, should no longer apply).
(I also improved some instructions quite a bit, but that's just a content
change, not really a templating change).
I made all the adjustments on the staging wiki (I hope it's not getting
purged over night). To compare the changes, click on the testcases from the
production wiki:
https://fedoraproject.org/wiki/Template:Installation_test_matrix#Upgrade
and from the staging wiki:
https://stg.fedoraproject.org/wiki/Template:Installation_test_matrix#Upgrade
See how the templates are used. The basic structure is so short and simple
that I'll quote it here:
{{QA/Test_Case
|description={{upgrade_description|from=current|flavor=workstation|pkgset=workstation|live=1|tool=gnome-software}}
|setup={{upgrade_setup|from=current|flavor=workstation|pkgset=workstation|live=1}}
|actions={{gnome_software_upgrade_procedure}}
|results={{upgrade_results}}
}}
I think it's easy to see which template to look at if you want to adjust a
particular section. Also it's quite obvious which parameters are passed to
those templates and what effect it has.
All those test cases are composed from these templates:
https://stg.fedoraproject.org/wiki/Template:Upgrade_description
https://stg.fedoraproject.org/wiki/Template:Upgrade_setup
https://stg.fedoraproject.org/wiki/Template:Dnf_upgrade_procedure
https://stg.fedoraproject.org/wiki/Template:Gnome_software_upgrade_procedure
https://stg.fedoraproject.org/wiki/Template:Upgrade_results
Only the first one is somewhat complex (but short!). The rest is pretty
simple, the last 3 are completely unparametrized.
With this setup, I believe we still hit the goal of avoid testcase content
duplication (all the real content exists in a single place only, a change
affects all testcases), and it's much easier for folks to adjust them now.
The basic QA/Test_Case structure needs to be copied to all existing
testcases, but it should be very stable and I don't think we need to fear
desynchronization (I'll watch all those pages for changes anyway).
Thoughts? Objections to putting it to production?
[1]
https://fedoraproject.org/wiki/QA:Testcase_upgrade_gnome-software_current...
[2]
https://fedoraproject.org/w/index.php?title=Template:Testcase_upgrade&...