On Wed, 2010-12-01 at 16:55 -0500, Doug Ledford wrote:
The comparison is 100% fair because it points out the fundamental
problem with the current policy: if you don't have a paid staff of
testers to make sure testing is done in a timely fashion, then you have
absolutely no business gating updates on a testing staff that doesn't
exist. It's nice in theory to think we can force testing of updates
prior to their release, but if the testing staff simply isn't there,
then you aren't improving the product, you're just stopping progress.
The gating is not on 'a testing staff'. The gating is on *testing*.
I want to say again that I'm not particularly wedded to the current
policy and I don't mind at all if it changes, but I think we need to be
careful of the mindset that says 'we can't enforce any standards in
Fedora because it's a volunteer project so we must just accept what
people are willing to give us'.
Even though packaging in Fedora is a volunteer process, we still have
fairly rigorous packaging guidelines and a review process. We don't just
accept any package someone turns up and submits. i.e., we're enforcing
standards of quality, despite this being an entirely volunteer effort
and no-one being compelled to show up and provide packages of a
The concept of having a policy requiring updates to be tested before
they're issued is really no different. I think one point where we've
fallen over is that it wasn't sufficiently well discussed / communicated
in advance that this testing wasn't just going to 'get done' by some
independent group and no-one else would have to worry about it, but
would require a lot of people to chip in. In the same way that there
isn't some separate independent group that does package reviews, it's
just all maintainers chipping in when they can. I think perhaps those
who supported and voted for the policy kind of assumed this would
happen, and many others weren't actually aware of it.
I do think that for update testing to work well going forward we need to
engage more groups with it and make it clear it's not something that
some separate QA group is just going to do for everyone and no-one has
to worry about it. We can get, and already have got, some enthusiastic
people to sign up to run updates-testing and provide testing feedback
for the packages they use anyway, but the concept of there being a
hardcore group of dedicated testers who will go out of their way to
install, configure and test software they wouldn't usually use is not
one that's likely to fly, I don't think.
When software is packaged it's reasonable to expect that someone,
somewhere, uses it; if they don't, it probably shouldn't be packaged. We
need to find those people and engage them in the testing process, and it
seems to me that the maintainers of packages are as well placed as
anyone to help find and engage their users in this process.
In many cases it's easier than that; a lot of packages are maintained by
more than one person. It's not only perfectly okay but more or less
*what we want to happen* for co-maintainers to sign up as proven testers
and test each others' updates. There's a bunch of people in the anaconda
group, for instance; it's perfectly fine for you all to sign up as
proven testers and test each other's code. The testing doesn't have to
come from some impartial outside body, all we need is a sanity check.
I don't really see any reason why *everyone* who's a packager shouldn't
also have signed up to be a proven tester by now. I'd like to ask if
anyone has a perception that it's a hard process to get involved in, or
if they got the impression that they *shouldn't* get engaged in it, or
something like that. Maybe we can improve the presentation to make it
clear that this really ought to be a very wide-based process.
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org