The new Update Acceptance Criteria are broken

[Adam Williamson]

On Fri, 2010-11-12 at 14:32 -0500, Tom Lane wrote:
> Till Maas <opensource at till.name> writes:
> > On Mon, Nov 01, 2010 at 10:09:17AM -0700, Adam Williamson wrote:
> >> I disagree. The evidence you cite does not support this conclusion. We
> >> implemented the policies for three releases. There are significant
> >> problems with one release. This does not justify the conclusion that the
> >> policies should be entirely repealed.
> 
> > It was brought to my attention that also current Fedora releases have
> > problems with delaying important security updates.
> 
> Quite.  In my little corner of the system, none of the last several
> mysql and postgresql updates have gone out with less than a seven-day
> delay, despite some of them being security updates (admittedly not
> high-severity ones, but still).  And the trend is downhill: out of the
> last nine such updates, five shipped with zero karma because not even
> one tester had got round to looking at them.  How does it help anyone
> to delay releases when no testing will happen?
> 
> It's absolutely crystal clear to me that we don't have enough tester
> manpower to make the current policy workable; it's past time to stop
> denying that.  I'd suggest narrowing the policy to a small number of
> critical packages, for which there might be some hope of it actually
> working as designed.

the policy is already differentiated between critpath and non-critpath
packages; critpath *require* testing, there's no 7-day clause.

it's worth noting that part of the point of the 7-day clause is to cover
'invisible testing'; even if people aren't posting feedback to Bodhi,
it's likely that if the update actually is broken, we will find out one
way or another within 7 days (some people will post negative feedback to
Bodhi but not positive; or we'll get notified on an ML, or forums, or
something).

it's also worth noting that this is a communal effort: we don't have a
big batch of testing robots who test whatever they're told to test.
(yet). It's going to work much better if developers take some
responsibility for getting their packages testing it. if you're
packaging something, presumably you know *somebody* who uses it: the
idea is that you can ask them to test it and provide the bodhi feedback,
not just rely on someone who runs fedora-easy-karma as a matter of
course providing feedback.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net