exclude people from giving karma?

[Adam Williamson]

On Thu, 2014-02-27 at 02:18 +0100, Kevin Kofler wrote:
> Miloslav Trmač wrote:
> > I fully agree with you testers giving +1 is not even close to proper
> > validation, but what alternative to get proper validation do you propose
> > as an improvement?  Dropping autokarma would replace broken validation
> > with *no* validation; that's not an improvement.
> 
> The proposal here is only to abolish AUTOkarma, i.e. to require the 
> maintainer to manually push the update even if it has enough karma. In other 
> words, remove the "[ ] Enable karma automatism" checkbox and the 2 
> thresholds that go with it from Bodhi. (Just ensure that non-critpath 
> updates can always be manually pushed once they get a karma of +1, because 
> last I checked, this was STILL broken in Bodhi and required changing the 
> stable threshold to 1 to be able to push the update.)
> 
> This is not about the policies to not allow the maintainer to push without 
> karma, though I happen to think those should also be abolished. (If you ask 
> why: Because I think the maintainer is better placed to judge the quality of 
> his/her update than an arbitrary number of people with nothing more than a 
> FAS account. Feedback from the testers should only be informative.)
> 
> But again, I think that even with no other policy change, just removing the 
> "karma automatism" misfeature from Bodhi would be an improvement.

So here's an idea I do want to float on devel@ before taking it to FESCo
(and run by the AutoQA folks), but how about this?

We don't currently 'enforce' the AutoQA depcheck and upgradepath tests
as we don't consider them reliable enough. OK. But are they at least
reliable enough that we could disable karma automatism if one fails,
with a note to the update submitter that they can push the update
manually or re-enable automatism if they find the failure is a false
negative?

We've had two unfortunate broken-depcheck updates in the past week, now
- libreoffice and dnf. In the libreoffice case, oddly, depcheck was
reported as "PASSED" even though the log shows all sorts of dep issues -
http://autoqa.fedoraproject.org/results/758667-autotest/virt06.qa/depcheck/results/libreoffice-4.2.1.1-.html - but depcheck did catch the dnf one:

 autoqa - 2014-02-24 23:19:06
AutoQA: depcheck test FAILED on i386. Result log:
http://autoqa.fedoraproject.org/report/1ccpl (results are informative
only) 
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net