Refining the update queues/process [Was: Worthless updates]

[Till Maas]

On Fri, Mar 05, 2010 at 01:46:34PM -0800, Adam Williamson wrote:

> Ah. You're looking at it on a kind of micro level; 'how can I tell this
> package has been tested?'

For a package maintainer it is especially interesting, whether the own
update has been tested.

> Maybe it makes it clearer if I explain more clearly that that's not
> exactly how I look at it, nor (I think) how the rest of QA sees it, or
> what the proposal to require -testing is intended to achieve. We're
> thinking more about 'the big picture', and we're specifically thinking
> about - as I said before - the real brown-paper-bag,
> oh-my-god-what-were-they-thinking kinds of regressions, the 'systems
> don't boot any more', 'Firefox doesn't run' kinds of forehead-slappers.
> What we believe is that requiring packages to go to updates-testing for
> some time improves our chances of avoiding that kind of issue.

Afaics, this misunderstanding is a big problem, e.g. my expectations of
updates testing also differ. Maybe you can add some more information to
the wiki about what QA for updates testing currently tries to ensure,
what it actually ensures and plans for the future. E.g. I always noticed
that there is not much karma given, but now that I wrote the script and
noticed that I provided more feedback within two days than the top
tester for F11 did until now, the visible test coverage was/is a lot
worse than I imagined.

> Obviously, the more testing gets done in updates-testing, the better.
> Hopefully Till's script will help a lot with that, it's already had a
> very positive response. But the initial trigger for the very first

I just did a quick evaluation. There are 384 updates in F12 updates
testing when I last ran fedora-easy-karma and only 108 (28%) received
any comment with karma != 0. For F11 its 34/272 (12.5%). I am curious to
to how these numbers have changed in a week. I hope then everyone from
the QA SIG is using the script to report feedback, so it will be save to
say that an update was not tested at all if it did not receive any
feedback.

Regards
Till

[1] https://admin.fedoraproject.org/updates/metrics/?release=F12
[2] https://admin.fedoraproject.org/updates/metrics/?release=F11
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20100305/ae7ee3a8/attachment.bin