Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

[Gerald B. Cox]

On Tue, Jul 12, 2016 at 7:47 PM, Siddhesh Poyarekar <sid(a)reserved-bit.com&gt;
wrote:

>  ... which is not enough.  The definition of 'generally
> functional' is
>  vague, as all of us agree and we have seen examples of that being
>  misused in the past.  Requiring devel to document their packages is
>  one step forward and requiring testers to provide more detailed
>  feedback on what they tested (so that devel can decide if that is
>  sufficient) should be the step after that.
>  
Instead of concentrating on testers, what about the packagers who don't
even test their
applications before throwing them over the wall to bodhi.  I've seen
packages that didn't even
get past a simple dnf requisite test because the requires statement in the
spec file was
screwed up.  Now we have packagers blaming bodhi because they can't read
the instructions,
or assuming that a tester hasn't adequately tested a package because it was
just released
by bodhi (ignoring the fact that a tester could have gotten that same
package days ago from
koji).

As others have already pointed out bodhi was never intended to be a robust
testing system.
It was intended to give the packager a high level indication of whether or
not there are issues.  There is
nothing wrong with "generally functional".  At least you're getting some
feedback.
If you're concerned about the quality of the testing, then simply raise the
karma limit, or turn off the auto push.

If you start putting up barriers to people providing feedback you'll soon
find you have none.  Which would
you rather have, an idea that many people have installed your package and
haven't noticed any issues or
a nice complicated test plan that no one bothers to run and you are left
with no feedback.  If anything, if
there is a test script, that script should be run by the packager BEFORE
they send their application to
bodhi.  That would be more helpful.