Silly bodhi karma games

[Jonathan Calloway]

+1

I suggest giving us karma testers more in the way of test cases.  If it is possible to automate tests by scripting them, perhaps you could even add some elements of integration testing here.  I generally skip past updates that there is either not a test case for, something I've never heard of, or something I have no idea how to test.  I don’t' think it's fair to anyone to give positive karma for some library I can't even run a script or something against.

<JC>



-----Original Message-----
From: test-bounces at lists.fedoraproject.org [mailto:test-bounces at lists.fedoraproject.org] On Behalf Of Kamil Paral
Sent: Friday, March 20, 2015 4:56 AM
To: For testing and quality assurance of Fedora releases
Subject: Re: Silly bodhi karma games

> I think it can be a judgement call on certain packages. For example, I 
> maintain the Review Board packages which almost never get karma from 
> more than one person (and that usually only for whichiver Fedora or 
> EPEL branch that person is currently deploying to). Even at karma 1, 
> most Review Board packages sit in updates-testing until the timeout 
> passes.

Just one note - the number of people giving karma is not the same as the number of people actually using that proposed package build. For example, some of system-critical libraries don't usually receive too much karma, because people are not sure how to test them, and whether running app X or Y is sufficient to test them. So they don't report any feedback for them. But if they have updates-testing enabled, they still test them at least unknowingly. If those libraries got broken, they would know it very fast, and bugs would get discovered. So, in this case, we have almost no positive feedback "things work", but the important thing here is the absence of the negative feedback. And that's the purpose of the timeout.

This also applies to many leaf packages. We have many more people running with updates-testing than people regularly giving feedback to everything they have installed (my personal guess would be by several orders of magnitude). Even if the leaf package doesn't receive any karma, that timeout interval is very useful, because it gives people time to report issues, if they spot any. From my QA point of view, that timeout might be even more important than the feedback provided. And I have been very angry about some critical path packages pushing to stable with just 10 karma in _one or two days_. There are millions of Fedora users, with various hardware and software combinations - we can't afford to push something like kernel, mesa or X to stable if only 10 users give it a thumbs up. We need to let those hundreds or thousands of people running with updates-testing to install it as well, and give it the invisible thumbs up, which is not present in bodhi, but which is expressed by the lack of critical issues reported in bodhi, bugzilla or on mailing lists. And the best way to achieve that at the moment is to make the updates sit it updates-testing for at least a certain time.
--
test mailing list
test at lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test