Update testing policy: how to use Bodhi

Kevin Kofler kevin.kofler at chello.at
Sat Mar 27 02:50:07 UTC 2010


Adam Williamson wrote:
> There is one type of feedback we don't really want or need to capture:
> "I have tried this update and it doesn't fix bug #XXXXXX", where the
> update doesn't claim to fix that bug. This is a quite common '-1' in the
> current system, and one we should eliminate.

Indeed, those are clearly invalid -1 comments and need to stop.

> I think Bill's proposed policy can be modified quite easily to fit this.
> All it would need to say is that for 'important' updates to be accepted,
> they would need to have one 'type 1' feedback from a proven tester, and
> no 'type 2' feedback from anyone (or something along those lines; this
> isn't the main thrust of my post, please don't sidetrack it too
> much :>).
where
> 2. I have tried this update in my regular day-to-day use and seen a
> regression: bug #XXXXXX.

The problem is that type 2 feedback can also be invalid, e.g.:
* the regression may actually be caused by another update which is not part 
of the update group being commented on,
* the regression may be caused by an earlier update which is already stable, 
the user just didn't notice it right away (so this is actually a "doesn't 
fix a bug which the update doesn't claim it fixes" situation, not a 
regression; delaying or blocking the update does nothing to fix the 
regression which already slipped through to stable),
* the "regression" may actually be an issue with:
- yum or PackageKit (usually already fixed, but the user needs to update yum 
or PackageKit first),
- the user's setup (invalid repository configuration, attempts to mix 
different versions of a multilib package etc.),
- mirroring or some other networking issue completely unrelated to the 
update,
etc.

IIRC, we've had all of these happen for KDE updates.

In addition, there is a potential for this feature being abused for DoS 
attacks in the future, especially if it gets such a big significance.

So I don't think blocking an update outright for having received type 2 
feedback is sane at all.

        Kevin Kofler



More information about the test mailing list