Example of karma not being functional [Was:POSTUN scriptlet failure in rpm package cyrus-sasl]

[Adam Williamson]

On Thu, 2010-03-11 at 16:35 +0100, Ralf Corsepius wrote:

> This case is a nice example demonstrating several defects in "applying 
> karma votes for QA":
> 
> 
> 1. The update package was sitting in "updates-testing" since 2010-02-22.
> 
> 
> 2. It did receive +3 karma points before being pushed to "updates"
> c.f. https://admin.fedoraproject.org/updates/cyrus-sasl-2.1.23-6.fc12
> => There are people who claim to have tested it and not having noticed 
> anything unusual.

They didn't claim that. They claimed that it 'works fine here' and
'works fo[r] me' (the other message just says 'thanks', but we can count
it as 'works for me' as that's what the radio button says).

The point is: this update *does* work. The error message is non-fatal.
The software works. So what they claim is correct. What you claim is
also correct.

What this highlights is, indeed, a defect, though - the same one I
raised at the FESco meeting: we don't have a definition of what exact
criteria a package should meet to get a +1. Should we vote down updates
which have non-fatal scriptlet errors? That question doesn't yet have a
clear answer. It doesn't, however, mean the initial testers were 'not
carefully enough', as you claim on the ticket. It just means you were
working to different criteria.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net