> We have this:
>
https://fedoraproject.org/wiki/QA:Testcase_Package_Sets_Default_Package_I...
>
> That covers GNOME. We could modify it and ask the tester to pick
> any
> one of the release blocking desktops to be installed. Do we want to
> make it fuzzy? (You won't know exactly what was tested). Or we can
> add
> a new KDE test case. Or we can leave it as it is. In this case, all
> variants seem OK to me. I would probably make it fuzzy or do
> nothing.
I think Josef's probably right here, the most straightforward thing
to
do would seem to be to just test it explicitly. It's in the criteria,
writing the test case(s) should be simple, and they're not onerous
tests
to do, so I don't really see a problem.
I think 'the default package set install works' is a Thing in itself,
so
I'd probably be more in favour of adding a separate test case which
covers installing each of the release-blocking desktops and can be
marked as 'pass' only if they all work. In practice, whoever's doing
the
test obviously can 'count' the default_package_install result towards
the release_blocking_package_sets_install (or whatever we call it)
test
case if the default package set at the time happens to correspond to
one
of the release-blocking desktops, as it does now.
So I'd imagine this would happen:
1. Tester checks that a default install works, marks
default_package_install test as pass
2. Tester checks that a KDE install works, and then marks
release_blocking_package_sets_install as pass
in the future if our default package set is somehow not a release
blocking desktop any more, or we get more release blocking desktops,
everything continues to work out, testers just have a bit more
testing
to do before marking release_blocking_package_sets_install as pass.
That is algorithmically perfect, but over-engineered [1]. I think it's much easier to
have a default_package_install and a KDE_package_install and adjust it if the defaults
changes, than to think about the correct workflow every time we execute the tests (I
understand I'll be fired for saying this, but I tend to forget these things. I also
want to make release validation more accessible for newcomers).
[1] $ python -c 'import this'