Update testing policy: how to use Bodhi
bochecha at fedoraproject.org
Fri Mar 26 23:11:19 UTC 2010
On Fri, Mar 26, 2010 at 23:49, Adam Williamson <awilliam at redhat.com> wrote:
[... snip ...]
> 1. I have tried this update in my regular day-to-day use and seen no
> 2. I have tried this update in my regular day-to-day use and seen a
> regression: bug #XXXXXX.
> 3. (Where the update claims to fix bug #XXXXXX) I have tried this update
> and found that it does fix bug #XXXXXX.
> 4. (Where the update claims to fix bug #XXXXXX) I have tried this update
> and found that it does not fix bug #XXXXXX.
> 5. I have performed the following planned testing on the update: (link
> to test case / test plan) and it passes.
> 6. I have performed the following planned testing on the update: (link
> to test case / test plan) and it fails: bug #XXXXXX.
This is basically what Doug had proposed, except that you added 5. and 6.
I know Luke also likes the idea, and I've been toying with
implementing Doug's idea in the Bodhi2 branch. Adding those 2 more
karma types shouldn't be too hard.
>Testers should be able to file multiple types of feedback in one
> operation - for instance, 4+1 (the update didn't fix the bug it claimed
> to, but doesn't seem to cause any regressions either). Ideally, the
> input of feedback should be 'guided' with a freeform element, so there's
> a space to enter bug numbers, for instance.
That's what I had in mind, but the Bodhi2 branch currently doesn't
have any controllers/template, so it will be some time before I have
something to show.
> I think Bill's proposed policy can be modified quite easily to fit this.
> All it would need to say is that for 'important' updates to be accepted,
> they would need to have one 'type 1' feedback from a proven tester, and
> no 'type 2' feedback from anyone (or something along those lines; this
> isn't the main thrust of my post, please don't sidetrack it too
> much :>).
That's actually one of the points I was missing : rules for
(auto-)push / (auto-)unpush. But yeah, it can be agreed on later.
> The system could do a count of how many of each type of feedback any
> given update has received, but I don't think there's any way we can
> sensibly do some kind of mathematical operation on those numbers and
> have a 'rating' for the update. Such a system would always give odd /
> undesirable results in some cases, I think (just as the current one
> does). I believe the above system would be sufficiently clear that there
> would be no need for such a number, and we would be able to evaluate
> updates properly based just on the information listed.
> What are everyone's thoughts on this? Thanks!
I totally agree that this would be far more desirable than the current
More information about the devel