Guide to setting karma thresholds?

Luke Macken lmacken at redhat.com
Mon Jun 13 17:21:44 UTC 2011


Excerpts from Kevin Fenzi's message of Mon Jun 13 12:49:43 -0400 2011:
> On Mon, 13 Jun 2011 10:43:42 -0500
> Michael Ekstrand <michael at elehack.net> wrote:
> 
> > I'm working on pushing my first bugfix to F15 (#711261), using the
> > guides I found in the wiki[1][2].  For a non-critical-path package,
> > the Update Policy says that it needs to meet the positive karma
> > threshold set by the submitter, but does not indicate what that
> > threshold should be or guidance for determining appropriate values.
> > The default is 3; I'm assuming that leaving it there is a reasonable
> > thing to do (and I won't be surprised if the 7-day criteria will be
> > hit first for this package).
> > 
> > However, I am still wondering: is there any guidance or policy
> > published on how to determine appropriate karma thresholds?  What
> > justifies increasing or decreasing the thresholds?  Userbase?  Impact
> > of change?
> 
> There isn't that I know of. ;) Perhaps this would be a good chance to
> try and draft one. In the end it's up to the maintainer, but there
> could be some ideas to help along. Off the top of my head: 

Yeah, we have yet to step back and really think about the defaults for
the karma thresholds, after having the +3/-3 defaults for so long. Some
maintainers set the values very low to decrease the amount of time their
update spends in testing, and some set the values really high (or
disable them) to ensure that their update doesn't change state without
mantainer intervention. It's designed to fit both maintainer styles.

> * Are the changes small/well contained (less karma)
> * Is this a security update with a single security change (less karma)
> * Is this a popular package with lots of testers (more karma)
> * Is this something that has been tested already by upstream or another
>   distro? (less karma)
> * Is this a bug that only affects a small number of users? (more karma)
> * Is there likely to be another update soon? (less if you want to try
>   and get this fix out now, more if you just wanted testing on this, to
>   be obsoleted by another update when ready). 

These are all sound suggestions, but it seems like they refer to the
stable karma threshold only. There are still a variety of scenarios for
setting different unstable threshold values. For example, for updates
that shouldn't cause *any* problems/regressions, setting an unstable
threshold to -1 would cause the update to back-out as soon as anyone
encounters a single issue. On the other hand, some updates, like the
kernel, are inevitably going to get a bunch of negative karma no matter
what so this wouldn't be ideal for that case.

Hopefully we can come up with and document some best-practices for this
based on maintainer experience.

luke


More information about the devel mailing list