QA's Package update policy proposal

[Adam Williamson]

On Tue, 2010-03-09 at 19:38 +0100, Michael Schwendt wrote:

> > Please provide details on what's mad about Kamil's proposal.
> 
> Here:
> 
> | The package updates must spend at least 14 days [1] in the
> | 'updates-testing' repository, or at least 7 days [1] provided they have
> | karma of at least 3 [1] and no negative feedback. Only after that the
> | package maintainer may submit them to the 'updates' repository.
> 
> Even under consideration of [1], it's the same system that doesn't work
> for me and won't work for me.

Again, the actual criteria used are completely up for discussion. They
don't even have to be to do with Bodhi karma, necessarily. It's just a
placeholder. The key point is just that we provide the option for there
to be a filter point between updates-testing and updates. The nature of
the filtering is not decided.

> Under some circumstances it would be justified to overrule negative
> feedback even. E.g. if it's a user who just wants to block a good update
> because of his pet peeve.

That's why Kamil quickly added the 'Exception process' section before
sending the proposal to this list, just to make it clear that we
envisage there would be an exception process for cases where the policy
should be circumvented. We didn't really get time to fill in the
details, but we did want it to be clear that there should be a process
for that.

> > The proposal isn't really about expanding testing activities, it's about
> > formally codifying how the updates process is actually supposed to work.
> 
> It isn't a snapshot of how it works currently, though. 

It's fairly close. As far as I can see, all it intentionally changes
from the current process is that it provides a review point prior to
acceptance into -testing. There actually already *is* a review point
prior to moving to -updates, but it's currently owned by rel-eng and is
not highly publicized, and very little gets rejected. It is there,
though: rel-eng explained this earlier in the threads, and explained
that they are unhappy with it being informal. I'm trying to find the
post, but there's just too much to wade through, sorry :(

I haven't seen much fundamental disagreement with this in principle,
during the discussion; the argument has been about what kind of criteria
should be used to review changes, and the proposal expressly doesn't
take a hard line on what they should be.

In case it's not entirely clear, the review point prior to acceptance
into -testing is intended to be used solely for automated testing for
really obvious and uncontroversial breakages, like dependency errors.
For instance, a fedora-packager update went into F-13 updates-testing
today with a dependency on a package that's not in the repositories.
What we aim to do in future is have AutoQA check for that kind of
obvious breakage and require it be fixed before the package gets into
updates-testing.

The review point between updates-testing and updates is for more
advanced and potentially manual testing, but again, that's entirely up
for debate. I don't *think* anyone would say the underlying principle
that we should allow for some kind of gatekeeping between
updates-testing and updates is a bad idea, but that's what the
discussion is for :) Do you think it's a bad idea to allow for this in
the policy? Or are you worried more about the actual criteria used for
the gatekeeping?

> It is an attempt
> at introducing and hardcoding a process, and I'm unhappy with the
> proposals so far as long as they are supposed to cover my packages and
> confine me into something that would be much more plausible for the
> critpath package set.

It would certainly be possible to have different gatekeeping criteria
for critpath and non-critpath updates, under the policy. That may well
be a good idea, especially at first. I'm sure it'll come up at the FESCo
meeting.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net