Re: F18 Criterions/Testcases interconnection
On Thu, 2012-10-25 at 18:47 -0400, Kamil Paral wrote:
> I think 'the default package set install works' is a
Thing in itself,
> so
> I'd probably be more in favour of adding a separate test case which
> covers installing each of the release-blocking desktops and can be
> marked as 'pass' only if they all work. In practice, whoever's doing
> the
> test obviously can 'count' the default_package_install result towards
> the release_blocking_package_sets_install (or whatever we call it)
> test
> case if the default package set at the time happens to correspond to
> one
> of the release-blocking desktops, as it does now.
>
> So I'd imagine this would happen:
>
> 1. Tester checks that a default install works, marks
> default_package_install test as pass
> 2. Tester checks that a KDE install works, and then marks
> release_blocking_package_sets_install as pass
>
> in the future if our default package set is somehow not a release
> blocking desktop any more, or we get more release blocking desktops,
> everything continues to work out, testers just have a bit more
> testing
> to do before marking release_blocking_package_sets_install as pass.
That is algorithmically perfect, but over-engineered [1]. I think it's
much easier to have a default_package_install and a
KDE_package_install and adjust it if the defaults changes, than to
think about the correct workflow every time we execute the tests (I
understand I'll be fired for saying this, but I tend to forget these
things. I also want to make release validation more accessible for
newcomers).
[1] $ python -c 'import this'
Well I see your point, but it kind of cuts both ways - Josef clearly got
confused by a few cases where we overload test cases to test several
different things, so you can argue that it's actually more 'accessible'
when we try to stick to 'one test case tests one thing'. But your
approach would achieve what we need to achieve indeed.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net