Proposed udpates policy change

[Toshio Kuratomi]

On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> This is the policy that I expect to be discussed during the Fesco 
> meeting tomorrow. This is entirely orthogonal to the ongoing discussions 
> regarding whether updates in stable releases should be expected to 
> provide features or purely bugfixes, and I don't see any conflict in 
> introducing it before those discussions have concluded.
> 
> Introduction
> ------------
> 
> We assume the following axioms:
> 
> 1) Updates to stable that result in any reduction of functionality to 
> the user are unacceptable.
> 
I disagree with this as an axiom with this phrasing.  Most updates are
supposed to increase functionality for the user.  However, any change in
code can lead to different bugs showing up which reduces functionality in
a different area.  There is a tradeoff that needs to be made here; labeling
all reductions in functionality as unacceptable as an axiom is unworkable.

If you s/unacceptable/undesirable/ you have a better axiom.  That
acknowledges that the fix in one piece of  functionality may be more
important than the regression in another.  However, that makes
for a weaker case for a stringent policy.

> 2) It is impossible to ensure that functionality will not be reduced 
> without sufficient testing.
> 
> 3) Sufficient testing of software inherently requires manual 
> intervention by more than one individual.
> 
This isn't entirely true either.  One person can manually evaluate the
impact of certain changes to certain pieces of software.  But this is less
of an issue than the first axiom as the number of packages that fit this
category is likely to be small.

> Proposal
> --------
> 
> The ability for maintainers to flag an update directly into the updates 
> repository will be disabled.
>
This is the foundation of the new policy.  It provides the means to enforce
any other changes. Sufficient or insufficient justification for this should
down below so I'll discuss that there.

> Before being added to updates, the package 
> must receive a net karma of +3 in Bodhi.
> 
This I do not agree with on several fronts.

1) Net karma is really less interesting than whether any people have used
the package and any negative karma that was given has comments that can be
used to evaluate whether a package should still go into the repos.  (for
instance, negative karma may be given for things that don't work, but didn't
work with the previous update.  Or negative karma may be given for something
that is less important than the bugs that are being fixed by the update).

2) There needs to either be a method besides karma for updates which
haven't received positive or negative karma or there needs to be
a commitment of time from FESCo or a group that FESCo delegates to test
updates that haven't received feedback so that karma can be acquired when
none of the consumers of the update use bodhi to leave karma.


> 
> It should be noted that this does not require that packages pass through 
> updates-testing. The package will appear in Bodhi as soon as the update 
> is available. If sufficient karma is added before a push occurs, and the 
> update is flagged for automatic pushing when the karma threshold is 
> reached, the update will be pushed directly to updates.
> 
This is a good note to have in the policy.  Thanks.


> It is the expectation of Fesco that the majority of updates should 
> easily be able to garner the necessary karma in a minimal space of time. 
> 
Could you clarify why FESCo expects this given past usage of karma in bodhi?
Other people have given counter examples of updates that have not received
any feedback.  We can have lmacken run a query for percentage of updates
have had +3 karma statistics if the anecdotes are insufficently convincing.
Or you can explain what you think will change (for instance, FESCo will
allocate time to testing and giving karma to all packages which do not have
negative karma) to make this expectation true.

> Updates in response to functional regressions should be coordinated with 
> those who have observed these regressions in order to confirm that the 
> update fixes them correctly.
>
This is a good idea -- one snag is that a regression often affects multiple
Fedora releases.  Currently karma is often only given to a single release
out of the multiple where the bug needs to be fixed.

If you decide that the regressions are so severe that known bugs should not
be fixed unless positive karma is applied on each branch separately then
this is what the policy is supporting.  However, I think that's an
unreasonable weighting of regression risk.  The fact that the package works
successfully on F-n reduces the number of potential places that a regression
could be present in the F-(n-1) update.  This should be taken into
consideration as part of evaluating whether the update should be pushed to
other releases than the one the bug reported explicitly tested on.

> At present, this policy will not apply to updates that are flagged as 
> security updates.
> 
The security team operates outside of the normal policy currently so this is
expected but there should also be a method for getting critical bug fixes
out the door.  Perhaps something like "if a maintainer requests a package be
pushed sooner because it is a critical bugfix, fesco members will test the
package and add positive or negative karma before the next push to satisfy
the karma requirement".

> The future
> ----------
> 
> Defining the purpose of Fedora updates is outside the scope of Fesco. 
>
Note that the purpose of updates is not a Board issue or an FPC issue so
it's either a FESCo issue or something for the individual packagers to
decide.

> However, we note that updates intended to add new functionality are more 
> likely to result in user-visible regressions,

This is untrue.  An invasive bugfix can cause user-visible regressions more
easily than new functionality that only touches on existing code in one
place (for instance, adding a menu entry).  Perhaps you could phrase it as:

"In many cases, adding functionality increases the complexity of code more
than fixing a simple bug.  More complexity is more likely to result in
a user-visible regression."

>
>
> and updates that alter ABI 
> or API are likely to break local customisations even if all Fedora 
> packages are updated to match. We encourage the development of 
> mechanisms that ensure that users who wish to obtain the latest version 
> of software remain able to do so, while not tending to introduce 
> functional, UI or interface bugs for users who wish to be able to 
> maintain a stable platform.
> 

On this note, kparal has been working on an updates policy as well since the
QA team needs to understand the workflow to understand where AutoQA and
other pieces fit into the picture.  I think he should bring that up for
further discussion at this point as another take on this.

kparal, care to post a link to your proposal?
-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/devel/attachments/20100308/1e1a46ea/attachment.bin