3:59 p.m.
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
It should be noted that this does not require that packages pass through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Updates in response to functional regressions should be coordinated with
those who have observed these regressions in order to confirm that the
update fixes them correctly.
At present, this policy will not apply to updates that are flagged as
security updates.
The future
----------
Defining the purpose of Fedora updates is outside the scope of Fesco.
However, we note that updates intended to add new functionality are more
likely to result in user-visible regressions, and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match. We encourage the development of
mechanisms that ensure that users who wish to obtain the latest version
of software remain able to do so, while not tending to introduce
functional, UI or interface bugs for users who wish to be able to
maintain a stable platform.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
4:09 p.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
We assume the following axioms:
[..]
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
Your axioms are obviously wrong. An update which simply bumped a
release number would have the same functionality. Since you claim
these are axioms -- self-evident truths that form the basis for
further argument, anything else you wrote is unsound. Sorry :-!
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
4:14 p.m.
On Mon, Mar 08, 2010 at 22:09:25 +0000,
"Richard W.M. Jones" <rjones(a)redhat.com> wrote:
Your axioms are obviously wrong. An update which simply bumped a
release number would have the same functionality. Since you claim
these are axioms -- self-evident truths that form the basis for
further argument, anything else you wrote is unsound. Sorry :-!
It could cause a conflict or a problem with obsoletes even without getting
rebuilt.
The rebuilt version of the package could break if other packages it uses
for the build have changed since it was last built.
4:17 p.m.
On Mon, Mar 08, 2010 at 10:09:25PM +0000, Richard W.M. Jones wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> We assume the following axioms:
[..]
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
Your axioms are obviously wrong. An update which simply bumped a
release number would have the same functionality. Since you claim
these are axioms -- self-evident truths that form the basis for
further argument, anything else you wrote is unsound. Sorry :-!
What guards you against changes in the buildroot, non-deterministic
compiler bugs, cosmic rays and the like? The point is to test the
binaries that are being pushed into the hands of the users.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
4:20 p.m.
On Mon, Mar 08, 2010 at 10:17:01PM +0000, Matthew Garrett wrote:
> > 2) It is impossible to ensure that functionality will not
be reduced
> > without sufficient testing.
> Your axioms are obviously wrong. An update which simply bumped a
> release number would have the same functionality. Since you claim
> these are axioms -- self-evident truths that form the basis for
> further argument, anything else you wrote is unsound. Sorry :-!
What guards you against changes in the buildroot, non-deterministic
compiler bugs, cosmic rays and the like? The point is to test the
binaries that are being pushed into the hands of the users.
And note that the point says _sufficient_ testing. If the change is just a
rebuild with bumped version number, sufficient testing may simply mean
verifying that nothing _actually_ changed.
--
Matthew Miller <mattdm(a)mattdm.org>
Senior Systems Architect -- Instructional & Research Computing Services
Computing & Information Technology
Harvard School of Engineering & Applied Sciences
5:35 p.m.
Matthew Garrett wrote:
What guards you against changes in the buildroot, non-deterministic
compiler bugs, cosmic rays and the like? The point is to test the
binaries that are being pushed into the hands of the users.
There's a point at which the probability of breakage is so low that it's a
waste of time and effort to even consider it. Software will never be perfect
anyway.
Kevin Kofler
4:21 p.m.
On Mon, 2010-03-08 at 22:09 +0000, Richard W.M. Jones wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> We assume the following axioms:
[..]
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
Your axioms are obviously wrong. An update which simply bumped a
release number would have the same functionality. Since you claim
It would be rebuilt from source, as all updates are. The build would not
be performed in the exact same circumstances as the original build. We'd
certainly hope that wouldn't change anything, but it's impossible to be
certain.
Your post is clearly nitpicking, but this is an important point; I've
certainly seen cases when I was at Mandriva where a rebuild didn't work,
whereas the initial build still did; various odd issues, such as build
environment changes, can cause this.
Actually, it happened to me once, if I recall correctly, with Kompozer.
The initial build I'd done a few months previously for a given Mandriva
release worked in Cooker (Rawhide equivalent). If you took the exact
same SRPM and rebuilt it with the then-current Cooker environment - even
though the old binary package still worked in that environment! - the
resulting binary package did not work at all.
Which only goes to illustrate that updates are tricky and *really,
really* anything can go wrong.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
4:23 p.m.
On Mon, 2010-03-08 at 22:09 +0000, Richard W.M. Jones wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> We assume the following axioms:
[..]
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
Your axioms are obviously wrong. An update which simply bumped a
release number would have the same functionality. Since you claim
these are axioms -- self-evident truths that form the basis for
further argument, anything else you wrote is unsound. Sorry :-!
Richard: I'm afraid, from bitter experience, I can dispute that
argument :(
The essence of my counterargument is that in the meantime an update
might have been pushed to the build root tag that affects the rebuild,
and you might "silently" lose functionality.
The most common example I can think of is when a test in a configure
script starts failing for some reason beyond the control of the person
building the update: a package can be rebuilt without changing its
source, but all functionality relating to the configuration test will
"silently" go away. (youch!)
Or if you want a more direct but hopefully less common example, what
happens if a bug has been introduced into the compiler since you last
rebuilt? All you're doing is bumping a release number, without changing
your sources, but you could end up in a world of (new) pain.
Hope this is helpful; FWIW I think we need better automated testing
around our updates process.
Dave
4:31 p.m.
On Mon, Mar 08, 2010 at 05:23:34PM -0500, David Malcolm wrote:
[...]
Hope this is helpful; FWIW I think we need better automated testing
around our updates process.
OK OK, it was half a joke. I agree that automated testing is the way
forward here. Hopefully AutoQA will help here. And we should trust
packagers, especially for non-critical-path packages. (Isn't that
what critical path is all about?)
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
4:18 p.m.
Matthew Garrett wrote:
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
Would that apply also to new packages being pushed as updates to stable
releases, or only to updates to existing packages?
Björn Persson
4:57 p.m.
On Mon, Mar 08, 2010 at 11:18:17PM +0100, Björn Persson wrote:
Matthew Garrett wrote:
> Proposal
> --------
>
> The ability for maintainers to flag an update directly into the updates
> repository will be disabled. Before being added to updates, the package
> must receive a net karma of +3 in Bodhi.
Would that apply also to new packages being pushed as updates to stable
releases, or only to updates to existing packages?
That is a good point. It's obviously the case that a new package can't
be less functional than the non-existent package it replaces, but
there's also a risk that a new package could break other packages that
are already installed - which is basically a way of me saying "I don't
know yet", and I hope that we can sort that out with some further
discussion. The real risk I see with blocking new packages is that
there's much less chance for them to get external testing.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
4:21 p.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
Before being added to updates, the package must receive a net karma
of
+3 in Bodhi.
[...]
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
I don't know what to say.
If Fesco is aiming at getting rid of all the pesky packagers maintaining low
profile packages: You're well on your way.
--
sven === jabber/xmpp: sven(a)lankes.net
4:31 p.m.
On Mon, 2010-03-08 at 23:21 +0100, Sven Lankes wrote:
> It is the expectation of Fesco that the majority of updates
should
> easily be able to garner the necessary karma in a minimal space of time.
I don't know what to say.
If Fesco is aiming at getting rid of all the pesky packagers maintaining low
profile packages: You're well on your way.
This is a proposal to be voted on by FESco. Anyone can present any
proposal they like to FESco. It's not FESco's expectation (or Fedora
policy) unless they accept it. Right now, it's Matthew Garrett's
expectation, not necessarily anyone else's :)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
4:45 p.m.
On Mon, 8 Mar 2010 23:21:45 +0100, Sven wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> Before being added to updates, the package must receive a net karma of
> +3 in Bodhi.
[...]
> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.
I don't know what to say.
Right now (and after sending my early reply), I feel dizzy. I hope to
not take another look at the fedora devel list folder until tomorrow
when it will contain hundreds of message. After those monster threads
last week, now this.
If Fesco is aiming at getting rid of all the pesky packagers
maintaining low
profile packages: You're well on your way.
Well said.
Also important, Matthew even fills a FESCo seat. Proposals like that are not
what I expect from FESCo members. I'm severely disappointing.
4:48 p.m.
On Mon, 8 Mar 2010 23:45:57 +0100, I wrote:
Also important, Matthew even fills a FESCo seat. Proposals like that
are not
what I expect from FESCo members. I'm severely disappointing.
s/disappointing/disappointed/
Only demonstrates that I would prefer to stay away from such crap. Perhaps it's
time to unsubscribe from the list and take more drastic action when such a proposal
would be approved by FESCo.
5:05 p.m.
On Mon, 2010-03-08 at 23:45 +0100, Michael Schwendt wrote:
Also important, Matthew even fills a FESCo seat. Proposals like that
are not
what I expect from FESCo members. I'm severely disappointing.
Just to even things out: I am very happy with this proposal, and it is
exactly what I expect from FESCo members.
One thing that could perhaps be added is a hint that maintainers which
have trouble getting sufficient karma for their updates could seek the
help of our QA team on fedora-test-list. Or not ?
Matthias
8:38 p.m.
On 03/08/2010 11:45 PM, Michael Schwendt wrote:
On Mon, 8 Mar 2010 23:21:45 +0100, Sven wrote:
> On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
>
>> Before being added to updates, the package must receive a net karma of
>> +3 in Bodhi.
>
> [...]
>
>> It is the expectation of Fesco that the majority of updates should
>> easily be able to garner the necessary karma in a minimal space of time.
>
> I don't know what to say.
Right now (and after sending my early reply), I feel dizzy. I hope to
not take another look at the fedora devel list folder until tomorrow
when it will contain hundreds of message. After those monster threads
last week, now this.
> If Fesco is aiming at getting rid of all the pesky packagers maintaining low
> profile packages: You're well on your way.
Well said.
Yes.
As others already said, the axioms this proposal is based on are wrong,
the conclusions are wrong, the technology being applied is flawed (karma
votes), ... this whole proposal is insane.
Also important, Matthew even fills a FESCo seat. Proposals like that
are not
what I expect from FESCo members.
This begs for a question: How many people participating to Fedora does
this FESCO represent? IIRC, the last election had a very low
participation, which in many democratic elections would have meant the
elections to have missed some quorums and the elections to be invalid.
I'm severely disappointing.
Ditto.
Ralf
Ralf
5:12 p.m.
On Mon, Mar 08, 2010 at 11:21:45PM +0100, Sven Lankes wrote:
If Fesco is aiming at getting rid of all the pesky packagers maintaining low
profile packages: You're well on your way.
So, no, that's not the intent and it's realised that this is a problem.
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while, and there'd undoubtedly going to be some
difficulty in getting updates for more niche packages through as a
result. If people have further suggestions for how we can increase the
testing base then that would be awesome, but the status quo really
doesn't seem sustainable.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
5:25 p.m.
Matthew Garrett wrote:
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while, and there'd undoubtedly going to be some
difficulty in getting updates for more niche packages through as a
result. If people have further suggestions for how we can increase the
testing base then that would be awesome, but the status quo really
doesn't seem sustainable.
Perhaps as a data-point, before Till's recent easy-karma script, I'm
not sure any update of the git package ever received enough karma to
be automatically moved to stable. It's not like git is a package no
one around here uses.
That said, I don't disagree with the goal of pushing packages through
updates-testing. I'm just unsure if lesser used packages will ever
get out of updates-testing. Maybe >= 3 karma or 2-3 weeks would
suffice?
--
Todd OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Some people are like Slinkies... not really good for anything, but you
still can't help but smile when you see one tumble down the stairs.
5:26 p.m.
On Mon, Mar 08, 2010 at 11:12:11PM +0000, Matthew Garrett wrote:
> If Fesco is aiming at getting rid of all the pesky packagers
maintaining low
> profile packages: You're well on your way.
So, no, that's not the intent and it's realised that this is
a problem.
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while,
And even if that would be in place (for example by counting
non-fas-account bodhi feedback - it isn't counted currently) we would
still have the case where a maintainer cannot fix something that is
obviously broken without a) pestering others to 'vote for it' or b)
claiming that it is a security fix.
and there'd undoubtedly going to be some difficulty in getting
updates
for more niche packages through as a result.
All I can say is that if your proposal is accepted, that will mean that
I can no longer do my job as a packager (of a few low-profile packages).
If others can live with only ever updating things in rawhide then that's
fine - that's not something I feel I can do and neither can I be
bothered to lobby others to +1 my updates (or create 2 or 3 fake fas
accounts to push the updates through on my own).
If people have further suggestions for how we can increase the
testing
base then that would be awesome, but the status quo really doesn't
seem sustainable.
First of all: Make sure a FAS account isn't required to give feedback
and second: don't try to punish the 98% of the packagers that are doing
the right thing to make sure the 2% screwing up are stopped.
--
sven === jabber/xmpp: sven(a)lankes.net
5:50 p.m.
On Mon, Mar 08, 2010 at 11:12:11PM +0000, Matthew Garrett wrote:
On Mon, Mar 08, 2010 at 11:21:45PM +0100, Sven Lankes wrote:
>
> If Fesco is aiming at getting rid of all the pesky packagers maintaining low
> profile packages: You're well on your way.
So, no, that's not the intent and it's realised that this is a problem.
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while, and there'd undoubtedly going to be some
difficulty in getting updates for more niche packages through as a
result. If people have further suggestions for how we can increase the
testing base then that would be awesome, but the status quo really
doesn't seem sustainable.
Sustainable doesn't really seem like the correct word.... We've been
doing
it this way for many years now and it's not as if updates skipping directly
to stable and thereby introducing more regressions is a bigger problem than
it has been in the past.
Really, the goal of this policy is to reduce the number of regressions as
the problem exists and we want to cut down on it.
The places where the policy fails are:
1) it doesn't address how to actually get more testing done
2) it doesn't balance risk of regression against the factors that are
prompting the update in the first place (bugfixes, user requested features).
Without those two pieces, this policy just outlines how to enforce that
a package does not get into the repositories without a trail of testing
having occurred. That doesn't provide any benefits to the packagers so it's
not going to get buyin.
-Toshio
1:43 a.m.
On 08/03/10 23:12, Matthew Garrett wrote:
On Mon, Mar 08, 2010 at 11:21:45PM +0100, Sven Lankes wrote:
>
> If Fesco is aiming at getting rid of all the pesky packagers maintaining low
> profile packages: You're well on your way.
So, no, that's not the intent and it's realised that this is a problem.
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while, and there'd undoubtedly going to be some
difficulty in getting updates for more niche packages through as a
result. If people have further suggestions for how we can increase the
testing base then that would be awesome, but the status quo really
doesn't seem sustainable.
Can I make a tangential suggestion here ?
I am on the side that quite a few packages need much more testing before
they appear in stable. Fedora, for me, has got less and less stable
and thus less usable. I have been bitten for many years, especially by
the Graphics issues :(
However, to get testing done it actually requires a good many users to
actually test the packages. This requires it to be easy for the user to do.
This is especially true for the Graphics system where testing is needed
on lots of different hardware.
I wonder if a change to the Fedora package build/release systems could
help here by more closely coupling Bodhi/Others with updates-testing.
Some ideas:
1. All packages in Bodhi automatically appear in updates-testing.
2. The karma (or stability :) ) value is stored in the RPM.
3. A link to the Bodhi information is also stored in the RPM.
4. yum/rpm is modified to support the idea of a
"karma"/"package stability level".
5. Users can change the "package stability level" they are willing to accept
overall or on a per package or package group basis.
6. Simple GUI package manager extensions could handle the extra info or a simple
to use "package-testing" GUI could be developed to handle this allowing
users
to feedback issues in an easy way.
7. There would be an easy way for users to backout the duff packages.
(emails back to the user when an updated package is available ?)
8. A link with upstream would be provided so any user generated feedback
would be sent to the upstream developers or be easily available by
them. Bugzilla would be integrated into this.
9. Bodhi information would include info on particular items to test.
This would mean that all packages are immediately available to end users
in an easy to use way. Users could decide how "frontier" they would like their
system. Links to the Bodhi information would directly available to users
allowing feedback of issues and backout the packages. It would also
be nice to have package groups such as
"Graphics: kernel,libdrm,mesa,xorg-x11-* etc) so that an entire group of
related packages could be karma'ed, installed and reverted in an easy way.
In the background the system can check for package dependency issues and
notify the package managers automatically. Obviously how the
"kama"/"package
stability level" is calculated is an issue. In fact updates-testing could
probably go and we would just have updates ...
Any views, anyone got some financial resources :)
Terry
2:41 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/09/2010 12:12 AM, Matthew Garrett wrote:
...
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while
.. but should happen before the actual policy change takes place. I'm
afraid the impact without proper arrangements won't only be frustrated
maintainers realizing how many weaknesses from parliamentarism (does
your acting reflect the grass roots' desires?) Fedora has to bear but
also loss of distribution superiority with many packages gone or
forsaken by never or far too late being pushed to stable.
QA enforcement works well for businesses were people get actually paid
for reviewing and testing stuff before going out to customers but since
Fedora is (mainly) a community-driven distribution this mechanism should
be provided opt-in / opt-out and enforced only where major system
breakage may occur, coincidentally this is where many users are actually
in place to give Karma (e.g. who doesn't use glibc?).
Why not provide a mechanism to count the approximate number of users for
a package first, coupling this with the minimum Karma automatism - where
low-profile packages are excluded from this by not reaching a minimum
threshold? This approach would be sophisticated, sensible and IMO easily
accepted by the majority of users and maintainers.
- --
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
iEYEARECAAYFAkuWCcQACgkQVTRddCFHw10KGgCglDxJA4U7hw0Q6LN39JT+Hv4C
TkoAoIwIKawBDPnXeFZr0eSkcl500z9C
=W34l
-----END PGP SIGNATURE-----
5:34 a.m.
On Mon, Mar 08, 2010 at 11:12:11PM +0000, Matthew Garrett wrote:
On Mon, Mar 08, 2010 at 11:21:45PM +0100, Sven Lankes wrote:
>
> If Fesco is aiming at getting rid of all the pesky packagers maintaining low
> profile packages: You're well on your way.
So, no, that's not the intent and it's realised that this is a problem.
We need to work on making it easier for users to see that there are
available testing updates and give feedback on them. This is clearly
going to take a while, and there'd undoubtedly going to be some
difficulty in getting updates for more niche packages through as a
result.
It seems to me that the problem is a lack of testing, not irresponsible
maintainers that don't care about testing. If the testing problem can be
solved to the point that a policy like this could work without
completely freezing updates for low profile packages then I think
maintainers would be keen to have the testing, and you won't need the
policy to force the issue.
Ewan
3:04 a.m.
On Mon, Mar 8, 2010 at 11:21 PM, Sven Lankes <sven(a)lank.es> wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> Before being added to updates, the package must receive a net karma of
> +3 in Bodhi.
[...]
> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.
I don't know what to say.
If Fesco is aiming at getting rid of all the pesky packagers maintaining low
profile packages: You're well on your way.
Very well said.
What i clearly miss is the Boards vision given to FESCo. Or am i wrong
here and it's FESCo who decides what will happen and the Board says,
well yeah, lets try that. We can always clean up and change it after
the train wreck? Or does the Board have nothing to do with it at all?
Well, a clean up isn't needed if it becomes a train wreck, it will be
already cleaned out.
Or with other words, is there a statement of the Board, officially
making clear what *they* want, with a link to it?
What FESCo want is clear meanwhile. By the way, is there some
documentation on how to vote out particular FESCo members and who can
do it? Just in case one thinks that a particular member of FESCo is
wrongly voted in. Or is that like with politicians, once elcected you
lost.
Same question for Board members.
Thanks in advance.
P.s. I hope it's clear that this is not trolling, but searching for
some clarity on the hierarchy and what the small developer can do if
he feels something is wrong.
--
LG Thomas
Dubium sapientiae initium
6:07 p.m.
On Tue, 9 Mar 2010 10:04:22 +0100
Thomas Janssen <thomasj(a)fedoraproject.org> wrote:
...snip...
By the way, is there some
documentation on how to vote out particular FESCo members and who can
do it? Just in case one thinks that a particular member of FESCo is
wrongly voted in. Or is that like with politicians, once elcected you
lost.
Same question for Board members.
FESCo talked about this at todays meeting.
See right around:
http://meetbot.fedoraproject.org/fedora-meeting/2010-03-09/fesco.2010-03-...
and also see the Board policy:
https://fedoraproject.org/wiki/Board/SuccessionPlanning
kevin
3:11 p.m.
On Mon, 2010-03-08 at 23:21 +0100, Sven Lankes wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> Before being added to updates, the package must receive a net karma of
> +3 in Bodhi.
[...]
> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.
I don't know what to say.
If Fesco is aiming at getting rid of all the pesky packagers maintaining low
profile packages: You're well on your way.
I usually stay away from mega-threads, but well put!
I doubt that even major bug fixes in any of my (small) packages, ever
got more than 1-2 karma votes. Many got zero - not even a vote by the
original bug report owner!
Why am I getting punished because some package didn't get enough testing
(due to the low visibility of update-testing?) before it was pushed into
-updates and caused breakage?
Either we (package maintainers) are qualified to make sane decisions
about our package or we are not. I don't really see a middle ground
here.
I wonder if it's not high time to return the distinction between core
and extra functionality.
- Gilboa
3:21 p.m.
On Wed, 2010-03-10 at 23:11 +0200, Gilboa Davara wrote:
Either we (package maintainers) are qualified to make sane decisions
about our package or we are not. I don't really see a middle ground
here.
Being qualified to do something does not mean that one always does it
perfectly. Almost everyone's qualified to drive, yet road traffic
accidents happen _all the time_. The people who built the LHC were no
doubt qualified to do yet, yet it turns out to be a bit broken. You can
pull examples from literally every sphere of human experience.
People make mistakes - even qualified people, even super-proficient
people who make far fewer mistakes than *most* people. This is why we do
testing.
You're behind the debate, in any case; Matthew's proposal was not
accepted by FESCo at the meeting. No proposal was fully accepted, but
FESCo asked everyone to go and work from Bill Nottingham's proposal
(which, if you look at it, is far more moderate) for further review next
week. But I thought it was important to make the general point. Being
qualified to do something does not mean that you will always do it
perfectly.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
3:56 p.m.
On Wed, 2010-03-10 at 13:21 -0800, Adam Williamson wrote:
On Wed, 2010-03-10 at 23:11 +0200, Gilboa Davara wrote:
> Either we (package maintainers) are qualified to make sane decisions
> about our package or we are not. I don't really see a middle ground
> here.
Being qualified to do something does not mean that one always does it
perfectly. Almost everyone's qualified to drive, yet road traffic
accidents happen _all the time_. The people who built the LHC were no
doubt qualified to do yet, yet it turns out to be a bit broken. You can
pull examples from literally every sphere of human experience.
People make mistakes - even qualified people, even super-proficient
people who make far fewer mistakes than *most* people. This is why we do
testing.
You're behind the debate, in any case; Matthew's proposal was
not
accepted by FESCo at the meeting. No proposal was fully accepted, but
FESCo asked everyone to go and work from Bill Nottingham's proposal
(which, if you look at it, is far more moderate) for further review next
week. But I thought it was important to make the general point. Being
qualified to do something does not mean that you will always do it
perfectly.
I just finished reading the fixed proposal (Or actually, I just finished
reading the full thread). Thanks for the head's up.
- Gilboa
4:44 p.m.
On Wed, Mar 10, 2010 at 01:21:45PM -0800, Adam Williamson wrote:
On Wed, 2010-03-10 at 23:11 +0200, Gilboa Davara wrote:
> Either we (package maintainers) are qualified to make sane decisions
> about our package or we are not. I don't really see a middle ground
> here.
Being qualified to do something does not mean that one always does it
perfectly. Almost everyone's qualified to drive, yet road traffic
accidents happen _all the time_. The people who built the LHC were no
doubt qualified to do yet, yet it turns out to be a bit broken.
The LHC is an interesting analogy; it certainly has problems that can be
picked out with 20:20 hindsight, but there was no way anyone could have
changed the processes in advance that would prevented them coming up in
the first place. When you're trying to build something complicated and
push the boundaries of what's been done before then mistakes are
inevitable. For all its faults the LHC is absolutely the best thing of
its type on the planet. Fear of making mistakes shouldn't stop us
building things like the LHC, and it shouldn't stop us building things
like Fedora either.
We already know how to build things that are safe, boring, and have been
done before. Someone's got to build the cool new stuff.
Ewan
4:59 p.m.
On Wed, 2010-03-10 at 22:44 +0000, Ewan Mac Mahon wrote:
On Wed, Mar 10, 2010 at 01:21:45PM -0800, Adam Williamson wrote:
> On Wed, 2010-03-10 at 23:11 +0200, Gilboa Davara wrote:
>
> > Either we (package maintainers) are qualified to make sane decisions
> > about our package or we are not. I don't really see a middle ground
> > here.
>
> Being qualified to do something does not mean that one always does it
> perfectly. Almost everyone's qualified to drive, yet road traffic
> accidents happen _all the time_. The people who built the LHC were no
> doubt qualified to do yet, yet it turns out to be a bit broken.
The LHC is an interesting analogy; it certainly has problems that can be
picked out with 20:20 hindsight, but there was no way anyone could have
changed the processes in advance that would prevented them coming up in
the first place.
This is certainly true. However, if they'd decided to build the whole
thing based on their first calculations, measuring once and cutting
once, and without doing any checking to make sure they were building
everything in a straight line, it probably would've gone even worse =)
you're right it was a bad example to pick, though, without further
explanation.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
5:16 p.m.
On Wednesday, March 10, 2010, 5:59:20 PM, Adam Williamson wrote:
>On Wed, 2010-03-10 at 22:44 +0000, Ewan Mac Mahon wrote:
> The LHC is an interesting analogy; it certainly has problems that can be
> picked out with 20:20 hindsight, but there was no way anyone could have
> changed the processes in advance that would prevented them coming up in
> the first place.
This is certainly true. However, if they'd decided to build the
whole
thing based on their first calculations, measuring once and cutting
once, and without doing any checking to make sure they were building
everything in a straight line, it probably would've gone even worse =)
you're right it was a bad example to pick, though, without
further
explanation.
--
Adam Williamson
Um... Adam, on that straight line thing. Last time I checked, they
were trying to make a very large perfect circle. **GRIN**
Some days Murphy is the only winner, no matter what we do!
5:40 p.m.
On Wed, 2010-03-10 at 18:16 -0500, Al Dunsmuir wrote:
On Wednesday, March 10, 2010, 5:59:20 PM, Adam Williamson wrote:
>>On Wed, 2010-03-10 at 22:44 +0000, Ewan Mac Mahon wrote:
>> The LHC is an interesting analogy; it certainly has problems that can be
>> picked out with 20:20 hindsight, but there was no way anyone could have
>> changed the processes in advance that would prevented them coming up in
>> the first place.
> This is certainly true. However, if they'd decided to build the whole
> thing based on their first calculations, measuring once and cutting
> once, and without doing any checking to make sure they were building
> everything in a straight line, it probably would've gone even worse =)
> you're right it was a bad example to pick, though, without further
> explanation.
> --
> Adam Williamson
Um... Adam, on that straight line thing. Last time I checked, they
were trying to make a very large perfect circle. **GRIN**
Some days Murphy is the only winner, no matter what we do!
Man, but you guys like to pick nits. =)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
4:14 p.m.
On Wed, Mar 10, 2010 at 9:11 PM, Gilboa Davara <gilboad(a)gmail.com> wrote:
> If Fesco is aiming at getting rid of all the pesky packagers
maintaining low
> profile packages: You're well on your way.
I usually stay away from mega-threads, but well put!
I doubt that even major bug fixes in any of my (small) packages, ever
got more than 1-2 karma votes. Many got zero - not even a vote by the
original bug report owner!
Why am I getting punished because some package didn't get enough testing
(due to the low visibility of update-testing?) before it was pushed into
-updates and caused breakage?
Either we (package maintainers) are qualified to make sane decisions
about our package or we are not. I don't really see a middle ground
here.
I thought I would add a few thoughts to this now long running thread.
Firstly I have been a long standing - since Fedora Core 1 - user of
Fedora, and in general Fedora Linux has served me well through many
generations of new installs, across not only my own machines but also
those of relatives/friends whom I have been trusted to convert to sole
Linux use, up till now very successfully.
There have been large changes in recent times (KDE 3.5->4, major
changes to graphics drivers including open source Radeon/nouveau,
major boot process updates, KMS, pulseaudio etc etc). We have survived
all of those largely unscathed. I would hope that the machines that I
run on behalf of other users will continue to serve them well through
yum updates whilst in normal production service. I am lucky that I
can run my main machines on a released and current version of Fedora
that I expect will not fail in a catastrophic way after normal updates
- but I do have available other non-critical machines on which I can
run alpha or beta ( or even rawhide occasionally) versions, or run
current releases but be prepared to take the risk of running
unreleased packages from koji before they even hit bodhi, and before
they reach testing repos. If these machines suffer in a major way it
is not a disaster and I can re-install at worst - but the main
production machines remain up and running (until recently that is!) I
am lucky since I can usually find what fix is needed and sort it out.
However Aunt Bessie can't and relies on people like me to fix their
machines when their email stops working and a message appears on their
screen saying something that is incomprehensible to her!
Fine if it is only Aunt Bessie that I have to fix - but if Uncle Bob,
Grandma Celine, Grandad David, and 25 other assorted friends, cousins
and relatives all find their email stops one morning then I am going
to be unable to do my dayjob if I spend all my time getting their
machines all fixed because an update broke their production systems.
At that level I would say that the update that caused that level of
failure is no longer acceptable as a released update. I know that we
are all human, and that occasionally we will all make a mistake (I do
too!) but there is a threshold beyond which a failure is really not
acceptable and once crossed there is a chance that users and testers
will be alienated and move elsewhere. In that event I think the
responsible person(s) should gracefully accept that a threshold has
been crossed and learn from flack that ensues, even that has arisen
from an upstream change that they were largely unaware would have
serious consequences.
Remember also that there are users who only have a single machine and
if that breaks then it is much harder for him/her to sort out the
problem if there is a loss of email functionality and/or loss of dns
(remember the dnssec issue) leading to loss of network connectivity
since getting the information required to fix it will need access to
the net - so critical packages do need to be identified and tested at
a more intense level than less critical packages. The kernel is also
clearly critical and when dependencies on X and graphics drivers could
break machines then that needs special consideration also.
In general packages and their maintainers do a good job and we get
regular excellent updated packages almost daily - what a service! -
users of other alternatives to linux certainly don't get that level of
provision or anything like it!
I know that there has been a lot of soul searching and a genuine
attempt to move forward - let's all keep level headed and try to be
constructive rather than destructive in trying to make for a better
Fedora. We have support that is truly up to date - let's keep it that
way, but also avoid really serious breakage on production releases.
--
mike c
4:25 p.m.
W dniu 08.03.2010 22:59, Matthew Garrett pisze:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
It should be noted that this does not require that packages pass through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Updates in response to functional regressions should be coordinated with
those who have observed these regressions in order to confirm that the
update fixes them correctly.
At present, this policy will not apply to updates that are flagged as
security updates.
The future
----------
Defining the purpose of Fedora updates is outside the scope of Fesco.
However, we note that updates intended to add new functionality are more
likely to result in user-visible regressions, and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match. We encourage the development of
mechanisms that ensure that users who wish to obtain the latest version
of software remain able to do so, while not tending to introduce
functional, UI or interface bugs for users who wish to be able to
maintain a stable platform.
How about packages that have few people use? I almost never got any
karma on the updates I prepare.
Julian
4:27 p.m.
On Mon, 8 Mar 2010 21:59:29 +0000, Matthew wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
Unless the fixes contained within an update are _more important_ than a
dropped feature.
E.g. if upstream has removed some "functionality" deliberately, and
upgrading to upstream's code is the only way to move forward.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
-1 silly
-1 ridiculuous
-1 bad
===
-3 total
I would be very unhappy with FESCo (and I was one who voted during the
election), if something like that were approved.
It should be noted that this does not require that packages pass
through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Your wording or FESCo's? In either case, I disapprove this strongly.
I have failed to get bodhi karma from bug reporters multiple times
before. It is beyond my time to pester bug reporters, so they would
vote inside bodhi instead of simply adding a comment in bugzilla.
In many cases (ABRT generated tickets), I cannot even get them to reply
in bugzilla. I release updates in return to
- problem reports found in non-Fedora places,
- crap I see in daily diffs I create for upstream projects,
- problems I find myself, which haven't reported by anyone else but
likely affect other users.
I don't want such updates to be held up by artificial hurdles.
5:17 p.m.
On Mon, Mar 08, 2010 at 11:27:04PM +0100, Michael Schwendt wrote:
On Mon, 8 Mar 2010 21:59:29 +0000, Matthew wrote:
> 1) Updates to stable that result in any reduction of functionality to
> the user are unacceptable.
Unless the fixes contained within an update are _more important_ than a
dropped feature.
E.g. if upstream has removed some "functionality" deliberately, and
upgrading to upstream's code is the only way to move forward.
In that kind of situation, I think the maintainer would need a very good
reason to push this to a stable release. We've arguably been there with
Thunderbird, and we saw how much trouble that caused.
> It is the expectation of Fesco that the majority of updates
should
> easily be able to garner the necessary karma in a minimal space of time.
Your wording or FESCo's?
My wording, to be voted on by Fesco.
In either case, I disapprove this strongly. I have failed to get
bodhi
karma from bug reporters multiple times before. It is beyond my time
to pester bug reporters, so they would vote inside bodhi instead of
simply adding a comment in bugzilla. In many cases (ABRT generated
tickets), I cannot even get them to reply in bugzilla. I release
updates in return to
- problem reports found in non-Fedora places,
- crap I see in daily diffs I create for upstream projects,
- problems I find myself, which haven't reported by anyone else but
likely affect other users.
I don't want such updates to be held up by artificial hurdles.
As I've said elsewhere, this is a problem that needs solving. But I
don't believe that it's a problem that's best solved by allowing people
to push directly to stable.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
12:41 p.m.
Matthew Garrett wrote:
As I've said elsewhere, this is a problem that needs solving. But
I
don't believe that it's a problem that's best solved by allowing people
to push directly to stable.
* What other solution do you propose?
* Your proposal is doing much more than just banning pushing directly to
stable (which by its own would already be annoying enough).
Kevin Kofler
1:01 p.m.
On Tue, 2010-03-09 at 19:41 +0100, Kevin Kofler wrote:
* Your proposal is doing much more than just banning pushing directly
to
stable (which by its own would already be annoying enough).
Actually his proposal would allow for pushing directly to stable,
provided the update ticket got enough karma before the push action.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
5:40 p.m.
Michael Schwendt wrote:
On Mon, 8 Mar 2010 21:59:29 +0000, Matthew wrote:
> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.
Your wording or FESCo's?
Definitely not mine, I can assure you! To me, that statement sounds more
like something out of The Onion than like something one can actually claim
with a straight face!
Kevin Kofler
4:32 p.m.
...snip...
Thanks for working on this Matthew!
A small issue:
- If the policy states +3 is needed, does that mean we are locking all
updates to require this amount, no more no less? This could be bad
for packages where the maintainer might want more testing. Perhaps it
should be 'no less than +3' ? or 'at least +3' ?
I personally like this idea (at least to try out and see how well it
works). I do think we should continue to address longer term update or
pace issues.
I would suggest people with feedback write up their feedback clearly
for this thread and avoid back and forth filibustering.
kevin
5:45 p.m.
On Monday 08 March 2010 05:32:01 pm Kevin Fenzi wrote:
...snip...
Thanks for working on this Matthew!
A small issue:
- If the policy states +3 is needed, does that mean we are locking all
updates to require this amount, no more no less? This could be bad
for packages where the maintainer might want more testing. Perhaps it
should be 'no less than +3' ? or 'at least +3' ?
I personally like this idea (at least to try out and see how well it
works). I do think we should continue to address longer term update or
pace issues.
I would suggest people with feedback write up their feedback clearly
for this thread and avoid back and forth filibustering.
kevin
As a maintainer I have seen several of my packages sit in updates testing for
over 2 weeks with no comments and no karma. In fact they sat so long I got
nag mail about not pushing them. Requiring a karma of +3 to push is just not
going to work by itself. If anything it should be 'An update cannot be pushed
to stable until either it reaches a Karma of +3 or it has been in updates
testing for 14 days with no negative karma."
Steven
--
=====================================================
Steven M. Parrish
-------------------------------------------------------------------------------------------------
gpg fingerprint: 4B6C 8357 059E B7ED 8095 0FD6 1F4B EDA0 A9A6 13C0
http://tuxbrewr.fedorapeople.org/
irc.freenode.net:
Nickname: SMParrish
Channels: #fedora-kde, #fedora-olpc, #fedora-edu, #sugar, #packagekit
5:53 p.m.
On Mon, 2010-03-08 at 18:45 -0500, Steven M. Parrish wrote:
As a maintainer I have seen several of my packages sit in updates
testing for
over 2 weeks with no comments and no karma. In fact they sat so long I got
nag mail about not pushing them. Requiring a karma of +3 to push is just not
going to work by itself. If anything it should be 'An update cannot be pushed
to stable until either it reaches a Karma of +3 or it has been in updates
testing for 14 days with no negative karma."
There is a chicken/egg problem here. Karma isn't required, ergo it
doesn't happen. If karma is required, we might see an increase in karma
registered, as well as an uptick in tools development to help provide
karma (we're already seeing this in F13, one could conclude that part of
that is because karma is required for critpath packages).
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
8:43 p.m.
On 03/09/2010 12:53 AM, Jesse Keating wrote:
On Mon, 2010-03-08 at 18:45 -0500, Steven M. Parrish wrote:
> As a maintainer I have seen several of my packages sit in updates testing for
> over 2 weeks with no comments and no karma. In fact they sat so long I got
> nag mail about not pushing them. Requiring a karma of +3 to push is just not
> going to work by itself. If anything it should be 'An update cannot be pushed
> to stable until either it reaches a Karma of +3 or it has been in updates
> testing for 14 days with no negative karma."
There is a chicken/egg problem here. Karma isn't required, ergo it
doesn't happen. If karma is required, we might see an increase in karma
registered, as well as an uptick in tools development to help provide
karma (we're already seeing this in F13, one could conclude that part of
that is because karma is required for critpath packages).
=> All you're going to see is further bureaucracy and bureaucratic
overhead, but you'r not going to see better package quality.
Ralf
2:50 p.m.
Ralf Corsepius <rc040203(a)freenet.de> writes:
=> All you're going to see is further bureaucracy and
bureaucratic
overhead, but you'r not going to see better package quality.
Precisely. Who knows the package best? The packagers. If so why there
is someone else to decide (karma, the system, whoever)? This is insane.
The one with most info shall decide, simple. It's like voting that the
Sun rise twice a day.
If the maintainter makes mistake after mistake, esp. with an important
package, and everyone suffer, well just make him pay. Not all of the
maintainers.
--
Krzysztof Halasa
3:19 a.m.
On Tue, Mar 9, 2010 at 00:53, Jesse Keating <jkeating(a)redhat.com> wrote:
On Mon, 2010-03-08 at 18:45 -0500, Steven M. Parrish wrote:
> As a maintainer I have seen several of my packages sit in updates testing for
> over 2 weeks with no comments and no karma. In fact they sat so long I got
> nag mail about not pushing them. Requiring a karma of +3 to push is just not
> going to work by itself. If anything it should be 'An update cannot be pushed
> to stable until either it reaches a Karma of +3 or it has been in updates
> testing for 14 days with no negative karma."
There is a chicken/egg problem here. Karma isn't required, ergo it
doesn't happen. If karma is required, we might see an increase in karma
registered, as well as an uptick in tools development to help provide
karma (we're already seeing this in F13, one could conclude that part of
that is because karma is required for critpath packages).
I think the 14 days with no negative karma is a good solution. At
least for the packages I maintain. That is the way I use it manually
at the moment, I push packages to stable once I get the nag mail. I
also started to do mayor updates only to rawhide, enhancements to F-12
and security/mayor bug-fix updates to F-11.
I maintain mainly php-qa packages. I never received any karma for
those. So far I haven't received a single bug report. I might be the
only person using those packages (I hope not - is there any way to
find out?).
I like the idea of easy karma scripts, but these will only help for
packages which are used every day by a number of persons which are
prepared to use update-testing.
This could mean that I will only add new versions to rawhide and
simply ignore any F-XX. This would save me a lot of time, but is
probably not what users are looking for.
Cheers,
Christof
4:32 p.m.
-1. Nay. NoWay. No thanks. Uh uh.
I could find little or nothing in your proposal to which I agreed... so
decided not to quote any.
I just registered at Fedoraforums.org and voted "adventurous" in
Adam's poll. Just to make sure my voice is heard, and not the
shouting of folks who think they know what I want, and want to limit
my choices for my own good.
I'm using Fedora, which means I want to run leading edge (or at worst current)
software. I do software development, and want to have access to the latest
and greatest without running Rawhide. I've been a Fedora user since FC3.
I have no interest in using RHEL on my systems. You appear to want to turn
Fedora into a RHEL clone. Centos does that well enough already.
If you don't want to update your own packages except in rawhide or
the development release at alpha level, that is your choice. Please do
not inhibit my choices by trying to outlaw the very thing that makes Fedora
vital for me as an end user.
By the way, you made a trivial error when you created the Subject line.
Since stuff like that happens with packages, I'd like packagers to
have the ability to correct problems. Similarly, if a package is
under active development I appreciate the ability to receive
enhancements in the service stream for supported releases.
Either fixes or enhancements should be reasonably well tested, and
well documented (including update comments). Radically incompatible
fixes/enhancements that inconvenience users and break running systems
are indeed problematic... why yes, I do run bind on my Fedora servers.
Al
4:38 p.m.
On Mon, 2010-03-08 at 17:32 -0500, Al Dunsmuir wrote:
-1. Nay. NoWay. No thanks. Uh uh.
I could find little or nothing in your proposal to which I agreed... so
decided not to quote any.
I just registered at Fedoraforums.org and voted "adventurous" in
Adam's poll. Just to make sure my voice is heard, and not the
shouting of folks who think they know what I want, and want to limit
my choices for my own good.
As Matthew expressly said, "This is entirely orthogonal to the ongoing
discussions regarding whether updates in stable releases should be
expected to provide features or purely bugfixes"
The proposal neither intentionally nor accidentally prevents people
pushing 'adventurous' updates. It just requires that all updates receive
a certain level of testing feedback to be accepted.
You may still think it's a bad proposal, but it's important to note that
it's not about restricting the types of updates that can be done. So
voting in the poll I started doesn't have much to do with *this*
proposal.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
5:18 p.m.
Matthew Garrett wrote:
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
Even if it already went to testing and sit there for ages? This will lead to
MANY updates being stalled in testing forever! Hardly any update is getting
+3 karma right now. Karma is completely useless in practice. People "gaming
the system" are going to be the only way updates can go out at all under
such a proposal.
Sure, we'd all love that kind of test coverage. It's just not practicable at
all. We need to be realistic.
In addition, it has been explained very clearly in the discussions on this
mailing list why the current karma system is a poor indicator of update
stability. Relying on it as the only way an update can go stable is just
insane.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
You gotta be kidding! Just look at the current update landscape to see how
far from the truth this is.
(And I also wonder with what authority you speak in the name of FESCo as a
whole. The fact that what you're saying is so clearly wrong makes this all
the more an issue.)
Kevin Kofler
12:13 p.m.
On Tue, 2010-03-09 at 00:18 +0100, Kevin Kofler wrote:
> It is the expectation of Fesco that the majority of updates
should
> easily be able to garner the necessary karma in a minimal space of time.
You gotta be kidding! Just look at the current update landscape to see how
far from the truth this is.
(And I also wonder with what authority you speak in the name of FESCo as a
whole. The fact that what you're saying is so clearly wrong makes this all
the more an issue.)
The proposal is for FESCo to adopt that position. It is not a statement
of current fact.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
5:28 p.m.
On Mon, Mar 8, 2010 at 4:59 PM, Matthew Garrett <mjg59(a)srcf.ucam.org> wrote:
The future
----------
Defining the purpose of Fedora updates is outside the scope of Fesco.
However, we note that updates intended to add new functionality are more
likely to result in user-visible regressions, and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match.
Thanks to you, Fesco and all developers working on this. There is
something that everyone seems to be missing on this track, and I'd
like to bring some attention to it.
Distros do integration work, and the main thrust of the QA work around
a release is that all the packages work nicely _together_, that all
the subtle interactions interact the right way.
Updates are always risky. There is no doubt that the updated package
worked fine for the maintainer, and yet we see updates bombing out
spectacularly relatively often. This is because pushing out a single
package update is what a maintainer does, but this "package focus"
undermines what the distro does -- _integration_.
Small, low-risk bugfix updates respect that distro-wide goal. Major
risky updates disregard that goal -- yes, a specific package is new!
improved! but the implicit social contrct with your end users ("a
nicely integrated OS") is broken because the time and effort to shake
out the subtle integration issues were skipped.
From an OLPC PoV, major updates are rather troubling. We put together
two OSs based on Fedora (XO and XS OSs). We test the integrated OS
quite a bit ("we" being an outstanding team of volunteers around the
world), and we override a few packages, some of them very tightly
coupled to the platform (that is -- likely victims of subtle
interactions that may go haywire on a major update).
Given that, wearing my "XS lead dev" hat, I hope that Fedora focuses
its "update" policy on the safe, sane, low-risk bugfixes. Otherwise,
downstream projects that do careful QA are between a rock and a hard
place -- between taking potentially exploding updates or ignoring
updates altogether... and missing out on important bugfixes.
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
6:27 p.m.
On Mon, Mar 08, 2010 at 06:28:45PM -0500, Martin Langhoff wrote:
On Mon, Mar 8, 2010 at 4:59 PM, Matthew Garrett
<mjg59(a)srcf.ucam.org> wrote:
> The future
> ----------
>
> Defining the purpose of Fedora updates is outside the scope of Fesco.
> However, we note that updates intended to add new functionality are more
> likely to result in user-visible regressions, and updates that alter ABI
> or API are likely to break local customisations even if all Fedora
> packages are updated to match.
Thanks to you, Fesco and all developers working on this. There is
something that everyone seems to be missing on this track, and I'd
like to bring some attention to it.
Distros do integration work, and the main thrust of the QA work around
a release is that all the packages work nicely _together_, that all
the subtle interactions interact the right way.
Updates are always risky. There is no doubt that the updated package
worked fine for the maintainer, and yet we see updates bombing out
spectacularly relatively often. This is because pushing out a single
package update is what a maintainer does, but this "package focus"
undermines what the distro does -- _integration_.
Small, low-risk bugfix updates respect that distro-wide goal. Major
risky updates disregard that goal -- yes, a specific package is new!
improved! but the implicit social contrct with your end users ("a
nicely integrated OS") is broken because the time and effort to shake
out the subtle integration issues were skipped.
There are seveal things to note, though.
* The dnssec update which prompted this policy was not actually a problem of
integration with the rest of the OS.
* The policy doesn't place any value on the types of updates that are made,
therefore it is only making an indirect change to integration. For
instance, KDE updates that have been talked about elsewhere would likely
still go throguh as they enjoy widespread testing as part of the KDE SIG.
>From an OLPC PoV, major updates are rather troubling. We put
together
two OSs based on Fedora (XO and XS OSs). We test the integrated OS
quite a bit ("we" being an outstanding team of volunteers around the
world), and we override a few packages, some of them very tightly
coupled to the platform (that is -- likely victims of subtle
interactions that may go haywire on a major update).
Given that, wearing my "XS lead dev" hat, I hope that Fedora focuses
its "update" policy on the safe, sane, low-risk bugfixes. Otherwise,
downstream projects that do careful QA are between a rock and a hard
place -- between taking potentially exploding updates or ignoring
updates altogether... and missing out on important bugfixes.
This is an incomplete look at the picture. If you stop large and complex
bugfix updates for fear of what they do to integration at the Fedora level,
then OLPC will still miss out on the important bugfixes in their derived
distros. If they pull in the bugfixes becausee they deem them worthy
enough, then why shouldn't Fedora have shipped them in the first place and
let OLPC exclude them?
The Fedora maintainers need to judge whether a large and complex update
makes the user's experience enough better (for instance, by fixing a large
number of bugs or enough important bugs) that they should perform the
update. OLPC maintainers can decide whether they have the same values as
the Fedora maintainer. In no case can either Fedora or the OLPC maintainer
abandon their evaluation of the risks and rewards of consuming updates --
it's just a matter of whether the downstream has to package the update
themselves or exclude the update that was provided.
-Toshio
5:31 p.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
I disagree with this as an axiom with this phrasing. Most updates are
supposed to increase functionality for the user. However, any change in
code can lead to different bugs showing up which reduces functionality in
a different area. There is a tradeoff that needs to be made here; labeling
all reductions in functionality as unacceptable as an axiom is unworkable.
If you s/unacceptable/undesirable/ you have a better axiom. That
acknowledges that the fix in one piece of functionality may be more
important than the regression in another. However, that makes
for a weaker case for a stringent policy.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
This isn't entirely true either. One person can manually evaluate the
impact of certain changes to certain pieces of software. But this is less
of an issue than the first axiom as the number of packages that fit this
category is likely to be small.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled.
This is the foundation of the new policy. It provides the means to enforce
any other changes. Sufficient or insufficient justification for this should
down below so I'll discuss that there.
Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
This I do not agree with on several fronts.
1) Net karma is really less interesting than whether any people have used
the package and any negative karma that was given has comments that can be
used to evaluate whether a package should still go into the repos. (for
instance, negative karma may be given for things that don't work, but didn't
work with the previous update. Or negative karma may be given for something
that is less important than the bugs that are being fixed by the update).
2) There needs to either be a method besides karma for updates which
haven't received positive or negative karma or there needs to be
a commitment of time from FESCo or a group that FESCo delegates to test
updates that haven't received feedback so that karma can be acquired when
none of the consumers of the update use bodhi to leave karma.
It should be noted that this does not require that packages pass through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
This is a good note to have in the policy. Thanks.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Could you clarify why FESCo expects this given past usage of karma in bodhi?
Other people have given counter examples of updates that have not received
any feedback. We can have lmacken run a query for percentage of updates
have had +3 karma statistics if the anecdotes are insufficently convincing.
Or you can explain what you think will change (for instance, FESCo will
allocate time to testing and giving karma to all packages which do not have
negative karma) to make this expectation true.
Updates in response to functional regressions should be coordinated
with
those who have observed these regressions in order to confirm that the
update fixes them correctly.
This is a good idea -- one snag is that a regression often affects multiple
Fedora releases. Currently karma is often only given to a single release
out of the multiple where the bug needs to be fixed.
If you decide that the regressions are so severe that known bugs should not
be fixed unless positive karma is applied on each branch separately then
this is what the policy is supporting. However, I think that's an
unreasonable weighting of regression risk. The fact that the package works
successfully on F-n reduces the number of potential places that a regression
could be present in the F-(n-1) update. This should be taken into
consideration as part of evaluating whether the update should be pushed to
other releases than the one the bug reported explicitly tested on.
At present, this policy will not apply to updates that are flagged as
security updates.
The security team operates outside of the normal policy currently so this is
expected but there should also be a method for getting critical bug fixes
out the door. Perhaps something like "if a maintainer requests a package be
pushed sooner because it is a critical bugfix, fesco members will test the
package and add positive or negative karma before the next push to satisfy
the karma requirement".
The future
----------
Defining the purpose of Fedora updates is outside the scope of Fesco.
Note that the purpose of updates is not a Board issue or an FPC issue so
it's either a FESCo issue or something for the individual packagers to
decide.
However, we note that updates intended to add new functionality are
more
likely to result in user-visible regressions,
This is untrue. An invasive bugfix can cause user-visible regressions more
easily than new functionality that only touches on existing code in one
place (for instance, adding a menu entry). Perhaps you could phrase it as:
"In many cases, adding functionality increases the complexity of code more
than fixing a simple bug. More complexity is more likely to result in
a user-visible regression."
and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match. We encourage the development of
mechanisms that ensure that users who wish to obtain the latest version
of software remain able to do so, while not tending to introduce
functional, UI or interface bugs for users who wish to be able to
maintain a stable platform.
On this note, kparal has been working on an updates policy as well since the
QA team needs to understand the workflow to understand where AutoQA and
other pieces fit into the picture. I think he should bring that up for
further discussion at this point as another take on this.
kparal, care to post a link to your proposal?
-Toshio
5:45 p.m.
On Mon, Mar 8, 2010 at 6:31 PM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.
>
This isn't entirely true either.
#3 is so true that is central to what distros are about. Upstream
probably released a good updated version, but the distro role is to do
the integration, and shake out all the bugs in those subtle
interactions between components.
Otherwise, distros could provide the installer + @base and for the
rest we could all grab RPMs from upstreams' websites.
The QA of the integrated set of packages at particular release is hard
and complex. That's what Fedora does, as a distro, and is central to
what Fedora is, and the implicit social contract -- these components
perform together in tune like a well-rehearsed orchestra.
If the trombonist buys a new and shiny trombone, great, but for the
piece he's playing tonight he'll have to play with the old trombone.
The new trombone may be slightly out of tune, or louder, or tinnier;
none of those things are bad per se, but it will ruin the combined
effort.
One person can manually evaluate
That's the "it works for me" attitude. Works for small software
projects with a couple users. Not for a hugely complex OS, one that
others use as the base for their own work.
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
6:39 p.m.
On Mon, Mar 08, 2010 at 06:45:49PM -0500, Martin Langhoff wrote:
On Mon, Mar 8, 2010 at 6:31 PM, Toshio Kuratomi
<a.badger(a)gmail.com> wrote:
>> 3) Sufficient testing of software inherently requires manual
>> intervention by more than one individual.
>>
> This isn't entirely true either.
#3 is so true that is central to what distros are about. Upstream
probably released a good updated version, but the distro role is to do
the integration, and shake out all the bugs in those subtle
interactions between components.
Otherwise, distros could provide the installer + @base and for the
rest we could all grab RPMs from upstreams' websites.
The QA of the integrated set of packages at particular release is hard
and complex. That's what Fedora does, as a distro, and is central to
what Fedora is, and the implicit social contract -- these components
perform together in tune like a well-rehearsed orchestra.
If the trombonist buys a new and shiny trombone, great, but for the
piece he's playing tonight he'll have to play with the old trombone.
The new trombone may be slightly out of tune, or louder, or tinnier;
none of those things are bad per se, but it will ruin the combined
effort.
> One person can manually evaluate
That's the "it works for me" attitude. Works for small software
projects with a couple users. Not for a hugely complex OS, one that
others use as the base for their own work.
If you had bothered to quote my complete proposal you would find that you
didn't have to bother writing your message. I'll pull a Jef and quote it:
> This isn't entirely true either. One person can manually
evaluate the
> impact of certain changes to certain pieces of software. But this is less
> of an issue than the first axiom as the number of packages that fit this
> category is likely to be small.
You're willing to say that if I update one of my packages that has a script
of 30 lines, is a leaf node, and the update is to give the script an
optional argument to print output to stdout instead of writing to a file
that I'm incapable of building that package and then QA'ing the package from
the update-testing repository?
I'm specific that this is not a major problem because the number of packages
that can fall into this category is small. But #3 is not a sterling example
of an axiom as there are packages in the repository where small changes can
be applied and tested for regressions by a single person.
-Toshio
6:55 p.m.
On Mon, Mar 8, 2010 at 7:39 PM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
You're willing to say that if I update one of my packages that
has a script
of 30 lines, is a leaf node, and the update is to give the script an
optional argument to print output to stdout instead of writing to a file
that I'm incapable of building that package and then QA'ing the package from
the update-testing repository?
Yes. And that is 0.001% of the package updates. In fact, skip the
noise of pushing that as an update, wait until something interesting
happens to roll it out.
There is no gain in rolling an rpm everytime. And there is a cost --
in many "cheap" resources (bandwidth, cpu burn in builders), and in
costly resources, like review time of your downstreams if they care
about their QA.
But #3 is not a sterling example
of an axiom
#3 is correct for 99.9% of the worthwhile package updates. Don't call
it an axiom if it bothers you to be unfair to a tiny portion of
updates.
But is a damn important point that is central to what a distro is.
cheers,
m
--
martin.langhoff(a)gmail.com
martin(a)laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
8 p.m.
On Mon, Mar 08, 2010 at 07:55:03PM -0500, Martin Langhoff wrote:
On Mon, Mar 8, 2010 at 7:39 PM, Toshio Kuratomi
<a.badger(a)gmail.com> wrote:
> You're willing to say that if I update one of my packages that has a script
> of 30 lines, is a leaf node, and the update is to give the script an
> optional argument to print output to stdout instead of writing to a file
> that I'm incapable of building that package and then QA'ing the package
from
> the update-testing repository?
Yes. And that is 0.001% of the package updates. In fact, skip the
noise of pushing that as an update, wait until something interesting
happens to roll it out.
There is no gain in rolling an rpm everytime. And there is a cost --
in many "cheap" resources (bandwidth, cpu burn in builders), and in
costly resources, like review time of your downstreams if they care
about their QA.
But if someone requests this feature because they need to use it in their
environment, then the risk::reward ratio is low enough to justify it.
> But #3 is not a sterling example
> of an axiom
#3 is correct for 99.9% of the worthwhile package updates. Don't call
it an axiom if it bothers you to be unfair to a tiny portion of
updates.
But is a damn important point that is central to what a distro is.
And once again you've failed to quote the part of my message where I say
that this affects a small number of packages. Thanks.
-Toshio
6:19 p.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
I agree with these.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I have some difficulty agreeing that +3 is "sufficient testing" for all
packages. I know a *lot* of people who used Gwibber when I maintained
it. It's probably the most used package I have ever maintained, and I
couldn't get anybody to test it without lobbying people in #fedora-docs
and on the Planet. +3 is too much for packages like these.
It's not the job of a package maintainer to campaign and say "hey, pay
attention to my package that only provides one Python function for a few
people who need it," let alone a nicely-done end-user application that a
lot of people use.
Maybe +3 karma is OK if we allow for a push straight to updates after
the old_testing period comes along. (Oh look, another new one of those
in my INBOX now.)
At present, this policy will not apply to updates that are flagged as
security updates.
Why? Don't those need testing too?
--
Ian Weller <ian(a)ianweller.org>
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
8 p.m.
On Mon, Mar 8, 2010 at 1:59 PM, Matthew Garrett <mjg59(a)srcf.ucam.org> wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Hmm. So. I have a package, perl-Moose, that has 4,667 tests run at
build time. It depends on perl-Class-MOP, which has 2,225 tests, and
it in turn depends on perl, which has 234,776 tests run at build. On
a future note, we're working on setting up smoke testing, so when we,
say, rebuild perl-Class-MOP we also run perl-Moose's tests.
If I rebuild perl-Moose, or really, any of these packages, what sort
of manual testing would you suggest we require before pushing the
update?
-Chris
--
Chris Weyl
Ex astris, scientia
7:56 a.m.
On Mon, Mar 08, 2010 at 06:00:20PM -0800, Chris Weyl wrote:
Hmm. So. I have a package, perl-Moose, that has 4,667 tests run at
build time. It depends on perl-Class-MOP, which has 2,225 tests, and
it in turn depends on perl, which has 234,776 tests run at build. On
a future note, we're working on setting up smoke testing, so when we,
say, rebuild perl-Class-MOP we also run perl-Moose's tests.
If I rebuild perl-Moose, or really, any of these packages, what sort
of manual testing would you suggest we require before pushing the
update?
Getting it onto users' machines. Testing that a library behaves as
documented isn't the problem - the risk is that there's consumers of
that library that depend on undefined and (thus) untested behaviour.
Making sure that some people who actually use this package install the
update reduces the risk that that happens.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
10:41 p.m.
Hi Matthew,
Thank you to you, Josh, and others for putting effort into these
discussions. And thanks for bringing this up in the meeting tomorrow.
Some comments inline below that might be useful, or not.
Before I get into what you wrote, can I also suggest asking Fedora users
what they want? We already have an announce list, and we have things
like firstboot, and the fedoraproject start page. It would be *trivial*
to post a survey up on the website that most users are going to see, for
the next few months, and collate actual responses from end Fedora users.
If they want rolling updates, so be it and some of us are "wrong". But
for goodness sake, let's actually ask the users about it.
On Mon, 2010-03-08 at 21:59 +0000, Matthew Garrett wrote:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
Indeed. I believe this is the number one problem right now. The problem
is that a single package update can introduce compound issues with other
packages that were unforeseen and untested, especially the latter. This
is an issue even for the "slow moving" Enterprise distros, and they have
entire teams of people paid to do very thorough testing before pushing
each individual update. Fedora can't quite do that, but I believe that
is therefore even more of a reason to slow down the update pace. After
all, there's a new release in 6 months, not years later (as would be the
case with an enterprise distro). I'd rather wait 6 months for some
feature than have to take time out of the day to fix a stable box.
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I believe that this is possibly too limited. Aside from the obvious
abuse potential (which will always exist no matter what happens) -
obviously someone could just hate the process and decide to have a
couple of others sign off on everything no matter what - I think it
should be necessary to have a cooling off period between pushing an
update, it being voted on, and it going out live. It doesn't have to be
weeks, but it should be long enough for the person who actually reported
some bug to test that it is fixed and for others who aren't able to
devote time every day to see the update. I suggest 3-5 days.
I also suggest /considering/ implementing rolling updates rather than
pushing everything to stable. By rolling updates, in this case I mean
implementing a technical means (and this is tricky with mirrors) by
which not every user will receive the update at once. Perhaps PackageKit
already does something with random deltas for non-security updates. I
usually do updates myself manually (I upgrade stable Fedora systems only
when I am certain to have time to reboot, and then I will bite the
bullet and apply many updates at once) so I haven't noticed that.
Defining the purpose of Fedora updates is outside the scope of Fesco.
However, we note that updates intended to add new functionality are more
likely to result in user-visible regressions, and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match. We encourage the development of
mechanisms that ensure that users who wish to obtain the latest version
of software remain able to do so, while not tending to introduce
functional, UI or interface bugs for users who wish to be able to
maintain a stable platform.
It might be worth /considering/ something like "volatile" on other
distros. I know that's mostly for anti-spam type stuff. But perhaps some
parallel stream similar to updates-testing where users can elect to have
new features if they want them, or retain the stable release (in which
case all they get is security fixes). Basically, fedora-updates becomes
literally a separate stream disjoint from the regular fedora stream.
Jon.
2:28 a.m.
On Mon, 08 Mar 2010 23:41:44 -0500 Jon Masters wrote:
I also suggest /considering/ implementing rolling updates rather
than
pushing everything to stable. By rolling updates, in this case I mean
implementing a technical means (and this is tricky with mirrors) by
which not every user will receive the update at once.
Hi Jon,
please do not overload the term 'rolling updates'. That's just
confusing. Perhaps 'staggered' would convey your intended meaning
better.
But what do you mean to accomplish with it?
I guess when an update with an unforseen serious regression gets pushed
to updates you want to give us a chance to react before too many users
download it.
When the broken dbus update fiasco happened it was frustrating to know
that people in one time zone after another upgrade to the broken
build even though the regression was already known. The fixed build may
have been done quickly, but because of the long time it takes from
building a package to have it appear in updates on mirrors there was
not much Fedora could do to limit the damage.
If it is not possible to shorten the minimal from-Koji-to-mirrors delay
to something on the order of one hour (wouldn't that be awesome?), then
perhaps there should be a way to blacklist known brown-paper-bag
updates in the metalinks (which are not affected by the delay). I
believe this would be more useful than staggered updates.
Michal
3:48 a.m.
On Tue, 2010-03-09 at 09:28 +0100, Michal Schmidt wrote:
On Mon, 08 Mar 2010 23:41:44 -0500 Jon Masters wrote:
> I also suggest /considering/ implementing rolling updates rather than
> pushing everything to stable. By rolling updates, in this case I mean
> implementing a technical means (and this is tricky with mirrors) by
> which not every user will receive the update at once.
please do not overload the term 'rolling updates'. That's
just
confusing. Perhaps 'staggered' would convey your intended meaning
better.
Agreed. Henceforth corrected because I didn't really want to go there.
If it is not possible to shorten the minimal from-Koji-to-mirrors
delay
to something on the order of one hour (wouldn't that be awesome?), then
perhaps there should be a way to blacklist known brown-paper-bag
updates in the metalinks (which are not affected by the delay). I
believe this would be more useful than staggered updates.
You got exactly what I meant. I also think Terry's suggestion of
stability levels can easily be encoded in metadata in the yum repo, if
anyone wanted to do that. Then it could dynamically change as people
voted in Bodhi, and I could set a threshold of something awesomely high
for applying updates, or say "wait until it's a week old first" :)
Jon.
9:26 a.m.
On Mon, Mar 08, 2010 at 11:41:44PM -0500, Jon Masters wrote:
I believe that this is possibly too limited. Aside from the obvious
abuse potential (which will always exist no matter what happens) -
obviously someone could just hate the process and decide to have a
couple of others sign off on everything no matter what - I think it
should be necessary to have a cooling off period between pushing an
update, it being voted on, and it going out live. It doesn't have to be
weeks, but it should be long enough for the person who actually reported
some bug to test that it is fixed and for others who aren't able to
devote time every day to see the update. I suggest 3-5 days.
Note -- in the policy as written, there's no possibility of abuse because
there's no definition of what karma +1/-1 means.
In a different thread someone asked whether we wanted people to simply +1/-1
if the update installed.
In another thread, adamw said that QA's bar for acceptance is very low (he
had some examples but I'd rather he speak up than I go try to find it in the
mailing list archives :-).
This needs to be rectified as well.
-Toshio
2:52 a.m.
Matthew Garrett píše v Po 08. 03. 2010 v 21:59 +0000:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
I think first should come the answer on "what problem is this proposal
trying to solve?". Is it the plain number of updates? Is it the quality
of updates? What updates failed?
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
Unfortunately this tries to use the "one size fits all" method, but this
can't work in the recent Fedora with the number of source packages
attacking the 10000 mark. And for sure there are packages with many
users (millions?) and on other side packages with few users (dozens or
even less). And as others said, sometimes it's hard to get even a
response from the reporter on his own bug ...
It should be noted that this does not require that packages pass
through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Updates in response to functional regressions should be coordinated with
those who have observed these regressions in order to confirm that the
update fixes them correctly.
Yes, this can again work for the mainstream packages and not for the
rest. And as result the only way to get a newer version of a
non-mainstream package will be only to upgrade the whole distro with a
waiting time up to 6 months. The new release contains both bugfixes and
new features, because the author can't maintain multiple branches at a
time and is doing releases every few months - one of the open source
principles was release early, release often.
So what are the options if the situation is so serious that something
must be done:
- start automated testing of updates - let the machine work, it can do a
lot
- divide packages into more tiers - add one(?) level between critpatch
and the rest that would require better testing (for example containing
libraries used by another package), some statistics how much are
packages installed would be needed
- start creating personal repos with updated stuff - and I always
thought this is the wrong way that is used by other distros and their
users ...
- improve the updates delivery mechanism so the user can easily consume
the regular updates and also test and comment the non-stable ones, this
would also require educating the users that there can be a newer version
in updates-testing when they are not satisfied with the stable one
- every package must have a QA person, well at least 3 ...
And yes, there are some not very nice options too ...
Dan
9:42 a.m.
On Tue, Mar 09, 2010 at 09:52:28AM +0100, Dan Horák wrote:
Matthew Garrett píše v Po 08. 03. 2010 v 21:59 +0000:
> This is the policy that I expect to be discussed during the Fesco
> meeting tomorrow. This is entirely orthogonal to the ongoing discussions
> regarding whether updates in stable releases should be expected to
> provide features or purely bugfixes, and I don't see any conflict in
> introducing it before those discussions have concluded.
>
> Introduction
> ------------
>
> We assume the following axioms:
I think first should come the answer on "what problem is this proposal
trying to solve?". Is it the plain number of updates? Is it the quality
of updates? What updates failed?
AFAICS, it's to prevent regressions from creeping into the updates repo
unnoticed. However, it's not clear what the problematic regression::wanted
bugfix ratio is.
Plain number of updates would probably be cut down by this proposal but
policy that addresses that directly is not present in the current document
-- that's been talked about on this list in other threads though.
-Toshio
3:51 a.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
This seems naive to me. My experience is that there are few people
willing to provide testing karma, even for relatively high-profile
packages. It took about three months to get as many people to submit
positive testing results for an httpd F-11 update in updates-testing
recently.
Regards, Joe
7:20 a.m.
On Tue, Mar 9, 2010 at 10:51, Joe Orton <jorton(a)redhat.com> wrote:
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.
This seems naive to me. My experience is that there are few people
willing to provide testing karma, even for relatively high-profile
packages. It took about three months to get as many people to submit
positive testing results for an httpd F-11 update in updates-testing
recently.
If the issues this update was solbing really bothered them, they would
have provided feedback earlier.
I maintain some niche packages that almost no one uses/no one would
provide karma for. But if I'm asked for a bugfix, and I do it, I want
the people requesting it to tell me that it indeed fixes the issue and
doesn't break anything else. Why would I bother if they don't care?
----------
Mathieu Bridon
7:32 a.m.
On Tue, Mar 09, 2010 at 02:20:20PM +0100, Mathieu Bridon wrote:
On Tue, Mar 9, 2010 at 10:51, Joe Orton <jorton(a)redhat.com>
wrote:
> On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
>> It is the expectation of Fesco that the majority of updates should
>> easily be able to garner the necessary karma in a minimal space of time.
>
> This seems naive to me. My experience is that there are few people
> willing to provide testing karma, even for relatively high-profile
> packages. It took about three months to get as many people to submit
> positive testing results for an httpd F-11 update in updates-testing
> recently.
If the issues this update was solbing really bothered them, they would
have provided feedback earlier.
Hah, right. Back in the real world, most users expect free cake and
complain if they don't get it. I would not be optimistic about
resetting expectations there. For most updates I do, I end up doing the
testing myself and ignorning automated karma pushes.
Regards, Joe
12:22 p.m.
On 03/09/2010 05:32 AM, Joe Orton wrote:
On Tue, Mar 09, 2010 at 02:20:20PM +0100, Mathieu Bridon wrote:
> If the issues this update was solbing really bothered them, they would
> have provided feedback earlier.
Hah, right. Back in the real world, most users expect free cake and
complain if they don't get it.
We re-educate. You can have your free cake but we won't frost it until
you've helped us make sure it's properly baked.
Josh
8:11 a.m.
On Tue, 9 Mar 2010 14:20:20 +0100, Mathieu wrote:
I maintain some niche packages that almost no one uses/no one would
provide karma for. But if I'm asked for a bugfix, and I do it, I want
the people requesting it to tell me that it indeed fixes the issue and
doesn't break anything else.
Hard to comment on as the scenario is too vague. The bugfix may be
trivial. The testing may be difficult. The added responsibility
requirements ("doesn't break anything else") are scary. The level
of participation the package maintainer asks for may be considered
too much by the user. The user may have moved on already. Do you want
to wait for the next user to find the same flaw?
Why would I bother if they don't care?
Because other users judge about "your" broken product without contacting
you. That may be ordinary users, who build and spread an opinion about
Fedora's quality. Or more advanced users, who would build from source
tarball themselves and tell their friends and contacts that it works while
Fedora is broken. Or article writers, who review the distribution and may
try "some niche packages", too. The bug report from _one_ user may be the
most you get. Be thankful that somebody has taken the time to contact you.
Once you've been informed about the problem, the user expects you to take
appropriate action. Or else he would not remain a user, but would join and
become a co-maintainer or take over the package. You are the one to offer
something (on top of what upstream offers). Hence users believe that you
have bigger interest than themselves in fixing things.
It may be false expectations from users towards package maintainers,
but it's reality.
4:26 a.m.
On 03/09/2010 03:29 AM, Matthew Garrett wrote:
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I don't see how we expect that for all packages to get enough karma and
while some of them can get feedback within the current infrastructure
and considering the wide variety of packages (niche libraries for
example) it is naive to believe that we are going to accomplish and
hence my counter points are:
* We need improvements in our infrastructure (easy karma is one avenue
but Pacagekit integration and other ways to get users to provide input
needs to be in place first)
* We need to consider what we need as exceptions to this rule or more
sensibly enforce this rule only in crit path packages initially
* If a time limit is considered as a alternative we need to document
ways to escalate and file a exception if necessary and again I would
recommend only consider enforcing it for crit packages first
As it current stands I am against this proposal
Rahul
5 a.m.
Sounds pretty sensible.
We should also keep in mind that one size does not fit all. While core
and widely used packages should have a more conservative update path,
some packages could benefit from faster release. karma mechanism +
feedback integration in PK looks like a total win for the latter.
Promoting the use of fedora-easy-karma among contributors (kudos to
Till Maas !) would be more effective than a half baked proposition
(hasty decisions are often bad ones).
2010/3/9 Rahul Sundaram <metherid(a)gmail.com>:
I don't see how we expect that for all packages to get enough karma and
while some of them can get feedback within the current infrastructure
and considering the wide variety of packages (niche libraries for
example) it is naive to believe that we are going to accomplish and
hence my counter points are:
* We need improvements in our infrastructure (easy karma is one avenue
but Pacagekit integration and other ways to get users to provide input
needs to be in place first)
* We need to consider what we need as exceptions to this rule or more
sensibly enforce this rule only in crit path packages initially
* If a time limit is considered as a alternative we need to document
ways to escalate and file a exception if necessary and again I would
recommend only consider enforcing it for crit packages first
As it current stands I am against this proposal
Rahul
4:28 a.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
You didn't explain (in your proposal) why we need this change. Do you
have any statistics about number of regressions and bugs that have
been introduced by untested/bad updates in F-11 and F-12?
I think all such changes should be always based on real experience and
statistics otherwise the change is premature optimization.
Introduction
------------
We assume the following axioms:
0) Fedora strongly depends on well-motivated and non-frustrated
maintainers and open source developers. We want to increment
number of responsible maintainers who are able to use common
sense. Our mission is to keep maintainers happy otherwise we
will lost them and then we will lost users and our good position
in Linux community.
Our goal is to use technical arguments and don't introduce
non-technical discussions in our mailing lists.
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Always when I see that someone is trying to introduce a new rule I
have to ask myself ... why so large project like kernel is able to
successfully exist for 20 years without a huge collection of rules?
Karel
--
Karel Zak <kzak(a)redhat.com>
6:55 a.m.
On Tue, 9 Mar 2010, Karel Zak wrote:
Always when I see that someone is trying to introduce a new rule I
have to ask myself ... why so large project like kernel is able to
successfully exist for 20 years without a huge collection of rules?
the kernel has one rule which ends up working very well. The kernel has
the "its Linus' (and a few other people's) way or the highway".
Now, if you'd like to see fedora adopt that rule, I'd be curious to see
the results.
-sv
7:10 a.m.
On Tuesday 09 March 2010 13:55:53 Seth Vidal wrote:
On Tue, 9 Mar 2010, Karel Zak wrote:
> Always when I see that someone is trying to introduce a new rule I
> have to ask myself ... why so large project like kernel is able to
> successfully exist for 20 years without a huge collection of rules?
the kernel has one rule which ends up working very well. The kernel has
the "its Linus' (and a few other people's) way or the highway".
Now, if you'd like to see fedora adopt that rule, I'd be curious to see
the results.
Yes, we don't have Linus here ;-) But usually I like his decisions - mostly
strongly technical ones based on arguments = less politics.
Jaroslav
-sv
--
Jaroslav Řezník <jreznik(a)redhat.com>
Software Engineer - Base Operating Systems Brno
Office: +420 532 294 275
Mobile: +420 731 455 332
Red Hat, Inc. http://cz.redhat.com/
7:51 a.m.
On Tue, Mar 09, 2010 at 02:10:58PM +0100, Jaroslav Reznik wrote:
On Tuesday 09 March 2010 13:55:53 Seth Vidal wrote:
> On Tue, 9 Mar 2010, Karel Zak wrote:
> > Always when I see that someone is trying to introduce a new rule I
> > have to ask myself ... why so large project like kernel is able to
> > successfully exist for 20 years without a huge collection of rules?
>
> the kernel has one rule which ends up working very well. The kernel has
> the "its Linus' (and a few other people's) way or the highway".
>
> Now, if you'd like to see fedora adopt that rule, I'd be curious to see
> the results.
It's not (only) about Linus. It's about working environment and
strong focus on technical things.
Please, read:
http://www.kernel.org/doc/Documentation/ManagementStyle
Yes, we don't have Linus here ;-) But usually I like his
decisions - mostly
strongly technical ones based on arguments = less politics.
Yes, less politics.
Karel
--
Karel Zak <kzak(a)redhat.com>
8:12 a.m.
On Tue, 9 Mar 2010, Karel Zak wrote:
It's not (only) about Linus. It's about working environment
and
strong focus on technical things.
Please, read:
http://www.kernel.org/doc/Documentation/ManagementStyle
> Yes, we don't have Linus here ;-) But usually I like his decisions - mostly
> strongly technical ones based on arguments = less politics.
Yes, less politics.
Less politics? Seriously?
man that's funny.
-sv
11:05 a.m.
On Tue, Mar 9, 2010 at 7:12 AM, Seth Vidal <skvidal(a)fedoraproject.org> wrote:
On Tue, 9 Mar 2010, Karel Zak wrote:
> It's not (only) about Linus. It's about working environment and
> strong focus on technical things.
>
> Please, read:
> http://www.kernel.org/doc/Documentation/ManagementStyle
>
>> Yes, we don't have Linus here ;-) But usually I like his decisions - mostly
>> strongly technical ones based on arguments = less politics.
>
> Yes, less politics.
>
Less politics? Seriously?
man that's funny.
Its human nature. If people discuss things and someone's proposal gets
dissed/not added it gets listed as "politics and bureaucracy", if it
gets accepted "its meritocracy at its best." You see this a lot of
times on linux-kernel list (or in the sub-email lists). You also see a
heck of a lot of politics about where a patch needs to be fixed up to
be included. It just doesn't look like Fedora politics because its not
written down and does not have to be relied on ever again. [If Linus
wants one week of patches to be in the form of haiku and the next in
the form of sonnets.. well thats what people will produce.]
--
Stephen J Smoogen.
Ah, but a man's reach should exceed his grasp. Or what's a heaven for?
-- Robert Browning
6:48 a.m.
Dne 8.3.2010 22:59, Matthew Garrett napsal(a):
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I usually decrease required karma to +-1, but I have never experienced
package pushed to stable because of karma. Does it mean that I shouldn't
bother to fix bugs in the released distros, because new release will
newer get out of updates-testing?
Please clarify.
Thank you,
Matěj
--
http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC
He uses statistics as a drunken man uses lamp-posts... for
support, rather than illumination.
-- Andrew Lang
8:26 a.m.
On Monday, 08 March 2010 at 22:59, Matthew Garrett wrote:
[...]
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
-1 to this. I've packaged a number of things that I know just one user of.
I have no idea how many people actually use my packages or how to reach
them. Therefore it will most likely be impossible for me to get +3 on each
update and they'll never get out of testing. Do you really want this?
If this gets passed I'll probably orphan most of my packages (i.e. the ones
that have very little users). There's no point in maintaining them in Fedora
under this bureaucracy.
Regards,
R.
--
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org | MPlayer http://mplayerhq.hu
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
9:56 a.m.
On Tue, Mar 09, 2010 at 03:26:15PM +0100, Dominik 'Rathann' Mierzejewski wrote:
-1 to this. I've packaged a number of things that I know just one
user of.
I have no idea how many people actually use my packages or how to reach
them. Therefore it will most likely be impossible for me to get +3 on each
update and they'll never get out of testing. Do you really want this?
No, and it's an entirely fair question. I don't expect this proposal to
be passed as is - there's going to be further discussion of the details,
and the necessary conditions for something to be considered sufficiently
tested are clearly something that we need to think about carefully. If
there's a tiny number of users then we still want the package to be
tested, but hitting +3 may be implausible.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
10:23 a.m.
On Tue, 9 Mar 2010, Matthew Garrett wrote:
On Tue, Mar 09, 2010 at 03:26:15PM +0100, Dominik 'Rathann'
Mierzejewski wrote:
> -1 to this. I've packaged a number of things that I know just one user of.
> I have no idea how many people actually use my packages or how to reach
> them. Therefore it will most likely be impossible for me to get +3 on each
> update and they'll never get out of testing. Do you really want this?
No, and it's an entirely fair question. I don't expect this proposal to
be passed as is - there's going to be further discussion of the details,
and the necessary conditions for something to be considered sufficiently
tested are clearly something that we need to think about carefully. If
there's a tiny number of users then we still want the package to be
tested, but hitting +3 may be implausible.
Thanks,
-sv
8:56 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/08/2010 10:59 PM, Matthew Garrett wrote:
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
- -1, I would have to orphan most of my packages and give up my current
plans of founding a Common Lisp SIG as it will die suffering the
chicken-and-egg problem if this gets approval.
I'd rather fork Fedora than accept this without massive modification
(see my first post in this thread).
- --
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
iEYEARECAAYFAkuWYYEACgkQVTRddCFHw12RvQCeM5P2LRGw8V6GazOn/3HU8pky
z1gAoKbM+V0VDXdK5MLcHj/298D7+1KW
=/bwH
-----END PGP SIGNATURE-----
9:27 a.m.
On Mon, 2010-03-08 at 21:59 +0000, Matthew Garrett wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I am strongly against this proposal if it is enforced on all the
packages as a whole. It might be acceptable for critical path packages
however low profile packages which are not installed on most of the
systems (and we have huge number of such packages now in Fedora) will
not get the testing no matter what magic you will cast on the users to
attract them to testing.
A reasonable compromise might be to allow the manual push to stable
after a week or so of the package living in the testing repository.
Somehow speeding up the process of getting the package to the hands of
users once it is entered into bodhi would be also appreciable.
--
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
Turkish proverb
9:47 a.m.
On Mon, 08.03.10 21:59, Matthew Garrett (mjg59(a)srcf.ucam.org) wrote:
The ability for maintainers to flag an update directly into the
updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
Two questions:
How do you plan to convince people to actually test packages? My
understanding right now is that folks who end up testing packages do so
exclusively because they themselves ran into the bug that is being
fixed. And as soon as that itch they are scratching is fixed, they
seldomly report back about this. (At least that is what I am
experiencing. In quite a few cases I know that people happily took
packages from bodhi but never reported back)
Making the user Karma logic the *only* path into the stable distribution
hence makes the distro depend on feedback by our users. And I
wonder if we really want to depend on that.
And secondly: trhere are many bugs that are only triggered in very
particular use cases or on very particular hardware. Nonetheless they
are bugs that are worth to be fixed. For those it is going to be hard to
get updates accepted simply because the number of people who could and
want to test this is minimal. (And I know what I am talking of, having
to deal with PA compatibility with a vast number of different audio
hardware and drivers).
All in all, I think the Karma logic is a good tool to speed up things,
it is not useful as the *only* path into the stable distribution.
Unless of course we create some kind of body whose sole job is to test
and provide karma to updates that are not otherwise tested by the
users. And which hence can be blamed if packages stay stuck in bodhi.
Because right now there is a substantial number of updates I push that
stay stuck in bodhi for really long times because not enough people
bother... And if in the future those packages will stay there forever
because I cannot push them myself than I'd be royally annoyed.
Lennart
--
Lennart Poettering Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/ GnuPG 0x1A015CC4
11:26 a.m.
On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
I don't think this is ideal.
I'm having déjà vu from a lot of these discussions & proposals -- we've
been here before. When I created bodhi, it initially disallowed pushes
directly to stable. This behavior... didn't sit well with The
Community, for a variety of reasons. As you can tell by the uproar in
every single one of these threads, if these same restriction are put in
place again we will lose a lot of important contributors.
Instead of re-implementing old controversial behavior, lets look at some
recent behavioral changes in bodhi and see if we can build on top of
those to solve these problems.
For example, I recently implemented various policy restrictions for
critical path/no frozen rawhide updates for pending releases. In order
for a critical path update to get marked as 'stable' for a pending
release, it must have at least +1 karma from releng/qa members, and at
least +1 from an authenticated user. These values, of course, are
configurable.
I think a much better solution would be to require similar critical path
policies, across *all* releases, not just pending ones, while still
allowing non-critpath packages to go directly to stable.
Requiring a static amount of karma across all updates is not going to be
sufficient, especially with the current karma system (which I think
needs improvement). Especially now with the easy-karma script, people can
+1 dozens of updates at a time, reaching +3 will be almost too easy for
certain packages, and yet not not easy enough for others. Having
configurable karma thresholds for different packages seems to be
sufficient, and it allows package maintainers to use their intuition to
tweak these thresholds accordingly to fit their workflow and tester
base. For example, the kernel maintainers disable karma automation
entirely, and one could argue that this flexibility is important.
Regarding the current karma system, I think Doug Ledford brought up some
good ideas in the earlier thread 'Bodhi karma feature request', and
I know other bodhi developers agree.
With an improved karma system, stricter critpath update handling, AutoQA
integration, and giving The User more fine-grained control over what
types of updates they actually want, I think we'll be much better off
than we currently are.
luke
12:36 p.m.
Luke Macken wrote:
For example, the kernel maintainers disable karma automation
entirely, and
one could argue that this flexibility is important.
We also systematically disable karma automatism for KDE updates. We found
the numeric karma to be a very poor indicator of the quality of the update.
Kevin Kofler
12:48 p.m.
On Tue, 2010-03-09 at 12:26 -0500, Luke Macken wrote:
I think a much better solution would be to require similar critical
path
policies, across *all* releases, not just pending ones, while still
allowing non-critpath packages to go directly to stable.
That is an acceptable fallback. But just for the record, I consider the
critical path on my desktop to include not just kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day. I don't
personally care if KDE is updated 20 times a week, or about other
packages that aren't installed by default being rebased often.
Jon.
1:01 p.m.
On Tue, Mar 09, 2010 at 01:48:42PM -0500, Jon Masters wrote:
On Tue, 2010-03-09 at 12:26 -0500, Luke Macken wrote:
> I think a much better solution would be to require similar critical path
> policies, across *all* releases, not just pending ones, while still
> allowing non-critpath packages to go directly to stable.
That is an acceptable fallback. But just for the record, I consider the
critical path on my desktop to include not just kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day.
Right, the critical path package list does contain the GNOME stack, cups-libs,
etc.
http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt
luke
1:43 p.m.
On Tue, 2010-03-09 at 14:01 -0500, Luke Macken wrote:
On Tue, Mar 09, 2010 at 01:48:42PM -0500, Jon Masters wrote:
> On Tue, 2010-03-09 at 12:26 -0500, Luke Macken wrote:
>
> > I think a much better solution would be to require similar critical path
> > policies, across *all* releases, not just pending ones, while still
> > allowing non-critpath packages to go directly to stable.
>
> That is an acceptable fallback. But just for the record, I consider the
> critical path on my desktop to include not just kernel/udev/modules/etc.
> but also GNOME, cups, and other things I use each day.
Right, the critical path package list does contain the GNOME stack, cups-libs,
etc.
http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt
So then great. I thought a lot of that was on there already, but didn't
have the link handy (bookmarked). Well then, to me, at least having a
very high bar on those would probably be sufficient for now at least.
Jon.
1:02 p.m.
On Tue, 2010-03-09 at 13:48 -0500, Jon Masters wrote:
On Tue, 2010-03-09 at 12:26 -0500, Luke Macken wrote:
> I think a much better solution would be to require similar critical path
> policies, across *all* releases, not just pending ones, while still
> allowing non-critpath packages to go directly to stable.
That is an acceptable fallback. But just for the record, I consider the
critical path on my desktop to include not just kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day. I don't
personally care if KDE is updated 20 times a week, or about other
packages that aren't installed by default being rebased often.
The current critical path does include most of that. You can see the
current critpath list at:
http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt
It's generated daily, it's not a static list. It does currently include
most of the important bits of GNOME, and cups-libs.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
1:02 p.m.
On Tue, 2010-03-09 at 13:48 -0500, Jon Masters wrote:
That is an acceptable fallback. But just for the record, I consider
the
critical path on my desktop to include not just
kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day. I don't
personally care if KDE is updated 20 times a week, or about other
packages that aren't installed by default being rebased often.
http://kojipkgs.fedoraproject.org/mash/branched-20100308/logs/critpath.txt is the current
critpath.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
1:07 p.m.
On Tue, Mar 09, 2010 at 01:48:42PM -0500, Jon Masters wrote:
On Tue, 2010-03-09 at 12:26 -0500, Luke Macken wrote:
> I think a much better solution would be to require similar critical path
> policies, across *all* releases, not just pending ones, while still
> allowing non-critpath packages to go directly to stable.
That is an acceptable fallback. But just for the record, I consider the
critical path on my desktop to include not just kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day. I don't
personally care if KDE is updated 20 times a week, or about other
packages that aren't installed by default being rebased often.
http://koji.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt
josh
1:10 p.m.
Matthew Garrett (mjg59(a)srcf.ucam.org) said:
Introduction
------------
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
I agree with the sentiments here.
Proposal
--------
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
However, I do wonder about some of the concerns about this being
a requirement for all packages. So, here's a counter-proposal/expansion.
If need be, each of these policies could be considered separately, although
they do stack on each other.
....
Proposal
--------
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
2) Updates that constitute a part of the 'important' package set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
Bill
1:34 p.m.
On 03/09/2010 02:10 PM, Bill Nottingham wrote:
Matthew Garrett (mjg59(a)srcf.ucam.org) said:
> Introduction
> ------------
>
> We assume the following axioms:
>
> 1) Updates to stable that result in any reduction of functionality to
> the user are unacceptable.
>
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
>
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.
I agree with the sentiments here.
> Proposal
> --------
>
> The ability for maintainers to flag an update directly into the updates
> repository will be disabled. Before being added to updates, the package
> must receive a net karma of +3 in Bodhi.
However, I do wonder about some of the concerns about this being
a requirement for all packages. So, here's a counter-proposal/expansion.
If need be, each of these policies could be considered separately, although
they do stack on each other.
....
Proposal
--------
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
2) Updates that constitute a part of the 'important' package set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
If it's not critpath, and it passes AutoQA, then let it go straight to
stable. There are those updates that don't need a bunch of testing.
But, by defining the critpath set, and requiring positive feedback on
critpath packages, you have covered our users asses on the really
important packages. By having autoqa in place, you've covered the vast
majority of brown paper bag problems on all packages, not just critpath.
For non critpath packages, the brown paper bag stuff is all we really
need to mandate I think.
As an example, I pushed an update direct to stable a couple weeks ago.
It was to fix a bug in an init script where my use of:
udevtrigger
udevsettle
resulted in just some deprecation warnings from udev. I replaced those
calls to calls of udevadm trigger and udevadm settle and then tested
that A) the noise was gone and B) they still worked. I tested this on
all releases before building. And that was the *only* change that I
made. This update did not deserve to go through testing, it was
pretested. I would have appreciated an autoqa check to verify that
nothing in the build system broken the new package (maybe a dep change
or something), but really the changes of even that were extremely remote
at best. So, I can see where some changes simply don't warrant a bunch
of strict rules. We can have these changes even in critpath packages,
but the whole point of critpath is that we've isolated a set of packages
that are so important that it's worth it to take a little extra time
just to make sure things are right.
So, I like the policy in as much as I like AutoQA and critpath requires
testing. But since you have both of those things, I think loosening up
and letting non-critpath packages do with just AutoQA is sufficient.
--
Doug Ledford <dledford(a)redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
1:40 p.m.
On Tue, 2010-03-09 at 14:10 -0500, Bill Nottingham wrote:
Proposal
--------
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
I agree with the idea, but it should be better phrased. It's not the way
the test is implemented that matters, but what's being tested. The fact
that the test is done by AutoQA is really neither here nor there.
It would be safer to explicitly define the types of tests we want all
updates to pass: set out exactly what the 'obvious' hurdles every
package pushed should get over are (i.e. dependencies should be sane).
*How* we choose to test that is really just an implementation detail.
2) Updates that constitute a part of the 'important' package
set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
This feels broadly correct to me at present, and that's what I'll say at
the meeting if input from non-FESCo members is okay. I think Matt's
proposal is far too much of a leap at present; it may be true that we
can drive much more feedback in Bodhi than we currently get, but we
should prove that *before* we push a policy that relies on it happening,
not after. I know there's a potential chicken/egg problem there, but we
haven't even really tried very hard yet.
It also has other issues that have already been discussed, that are
mainly shortcomings in the current Bodhi process. As many have pointed
out (and QA has basically accepted), relying on a simple overall Bodhi
'score' is pretty unreliable at present, because we have no really good
process for defining what positive and negative Bodhi votes actually
mean. It's just not a robust enough process to make all updates depend
on getting a hard-defined overall Bodhi score at present. It can go
wrong in both directions; updates that shouldn't get pushed can get +3
(the obvious example being a kernel that boots fine for 7 people and
breaks for 3 would score +4), and updates that should get pushed might
not get +3.
Right now we should limit reliance on Bodhi to only critpath packages at
most, and even for that we do need to tighten up Bodhi procedures quite
urgently. And it shouldn't rely on an arbitrary combined
positive/negative score.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
1:43 p.m.
On Tue, Mar 09, 2010 at 02:10:04PM -0500, Bill Nottingham wrote:
2) Updates that constitute a part of the 'important' package
set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
How about every package that's installed by default in the primary
spins?
Comments, questions, reasoned arguments? Part of me wonders if this
should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
This seems pretty sane.
--
Matthew Garrett | mjg59(a)srcf.ucam.org
1:49 p.m.
Bill Nottingham píše v Út 09. 03. 2010 v 14:10 -0500:
Matthew Garrett (mjg59(a)srcf.ucam.org) said:
> Introduction
> ------------
>
> We assume the following axioms:
>
> 1) Updates to stable that result in any reduction of functionality to
> the user are unacceptable.
>
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
>
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.
I agree with the sentiments here.
> Proposal
> --------
>
> The ability for maintainers to flag an update directly into the updates
> repository will be disabled. Before being added to updates, the package
> must receive a net karma of +3 in Bodhi.
However, I do wonder about some of the concerns about this being
a requirement for all packages. So, here's a counter-proposal/expansion.
If need be, each of these policies could be considered separately, although
they do stack on each other.
....
Proposal
--------
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
2) Updates that constitute a part of the 'important' package set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
Thanks Bill, this proposal is very similar to my "dump of ideas" posted
earlier today. The only thing I would like to improve (probably in
PackageKit) is the presentation of the content in updates-testing to the
users, to make it more visible and to encourage the testing. Something
like subscription to selected subset of packages that I want to be
notified about.
Dan
2:13 p.m.
On Tuesday, March 9, 2010, 2:49:00 PM, Dan Horák wrote:
Thanks Bill, this proposal is very similar to my "dump of
ideas" posted
earlier today. The only thing I would like to improve (probably in
PackageKit) is the presentation of the content in updates-testing to the
users, to make it more visible and to encourage the testing. Something
like subscription to selected subset of packages that I want to be
notified about.
Dan
I liked the visual presentation of download stats that Adam reference.
It may have been Ubuntu or Debian related.
I would like a tool that would present the list of installed packages,
and allow one to subscribe based on that. Also, bringing it up a
level to comps might reduce the user time. Perhaps basing the
testing of dependent package testing based on the owning packages
would simplify things too.
Al
2:13 p.m.
On Tuesday, March 9, 2010, 2:10:04 PM, Bill Nottingham wrote:
However, I do wonder about some of the concerns about this being
a requirement for all packages. So, here's a counter-proposal/expansion.
If need be, each of these policies could be considered separately, although
they do stack on each other.
...
Proposal
--------
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
This is a sane approach.
One problem with immediate implementation would be that all packages,
no matter how insignificant would need to have tests that could be
run. Some packages in categories such as firmware or cross-compilation
tools would require specialized hardware to test fully as part of the
build or subsequent AutoQA testing.
2) Updates that constitute a part of the 'important' package
set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
Also sane.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Again, sanity.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
I think you need to separate enhancements/etc to core components from
those for lower risk packages.
2:20 p.m.
On Tue, 2010-03-09 at 15:13 -0500, Al Dunsmuir wrote:
> 1) All updates (even security) must pass AutoQA tests.
> Rationale: If a package breaks dependencies, does not install, or
> fails other obvious tests, it should not be pushed. Period. Obviously,
> this proposal would not be enacted until AutoQA is live.
This is a sane approach.
One problem with immediate implementation would be that all packages,
no matter how insignificant would need to have tests that could be
run. Some packages in categories such as firmware or cross-compilation
tools would require specialized hardware to test fully as part of the
build or subsequent AutoQA testing.
What Bill's talking about when he refers to 'autoqa tests' are generic
tests which are concerned with package quality, not really the software
in the package: stuff like do the dependencies work, are there any clear
errors in the file lists. They can be run on any RPM package, the
software in the package doesn't really matter.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
2:28 p.m.
On Tuesday, March 9, 2010, 3:20:25 PM, Adam Willamson wrote:
On Tue, 2010-03-09 at 15:13 -0500, Al Dunsmuir wrote:
> > 1) All updates (even security) must pass AutoQA tests.
> > Rationale: If a package breaks dependencies, does not install, or
> > fails other obvious tests, it should not be pushed. Period. Obviously,
> > this proposal would not be enacted until AutoQA is live.
>
> This is a sane approach.
>
> One problem with immediate implementation would be that all packages,
> no matter how insignificant would need to have tests that could be
> run. Some packages in categories such as firmware or cross-compilation
> tools would require specialized hardware to test fully as part of the
> build or subsequent AutoQA testing.
What Bill's talking about when he refers to 'autoqa
tests' are generic
tests which are concerned with package quality, not really the software
in the package: stuff like do the dependencies work, are there any clear
errors in the file lists. They can be run on any RPM package, the
software in the package doesn't really matter.
--
Adam Williamson
That makes sense.
How about things like rpmlint? Perhaps that would have caught the
bind/dnssec problems where user configs were directly rewritten
without backup to rpmnew files.
2:34 p.m.
On Tue, 2010-03-09 at 15:28 -0500, Al Dunsmuir wrote:
> What Bill's talking about when he refers to 'autoqa
tests' are generic
> tests which are concerned with package quality, not really the software
> in the package: stuff like do the dependencies work, are there any clear
> errors in the file lists. They can be run on any RPM package, the
> software in the package doesn't really matter.
That makes sense.
How about things like rpmlint? Perhaps that would have caught the
bind/dnssec problems where user configs were directly rewritten
without backup to rpmnew files.
We have a tool called 'rpmguard' which is more or less a cousin of
rpmlint that looks at the differences between two versions of a package
and flags up critical changes. That's what will be implemented in AutoQA
and used at this 'test point'. See
https://fedorahosted.org/autoqa/wiki/rpmguard .
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
2:14 p.m.
On Tue, 2010-03-09 at 14:10 -0500, Bill Nottingham wrote:
....
Proposal
--------
Comments, questions, reasoned arguments? Part of me wonders if this
should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
I've thrown together an extremely quick-n-dirty combination of Bill's
proposal and Kamil's here:
https://fedoraproject.org/wiki/User:Adamwill/Proposal:Package_update_policy
the wording is very rough, but I hope it's good enough to form a basis
for discussion.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net
2:36 p.m.
Matthew Garrett <mjg59(a)srcf.ucam.org> writes:
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
True.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Definitely. IOW, the testing is never sufficient.
1) Updates to stable that result in any reduction of functionality
to
the user are unacceptable.
That means any and all updates are unacceptable.
--
Krzysztof Halasa
2:48 p.m.
Hello Krzysztof,
Tuesday, March 9, 2010, 3:36:43 PM, you wrote:
Matthew Garrett <mjg59(a)srcf.ucam.org> writes:
> 2) It is impossible to ensure that functionality will not be
reduced
> without sufficient testing.
True.
The whole point of an update may be the deliberate removal of
features/functionality. This includes removal of elements whose
upstream is dead, removal of conflicts between packages, conversion of
static/copied libraries to the system provided library, removal of
features which are irretrievably broken, and those elements which do
not fit within Fedora's licence/mission (mp3 support, or patented
material).
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.
Definitely. IOW, the testing is never sufficient.
Any nontrivial piece of software contains bugs until it reaches
end-of-life. This is a simple fact of life.
You can't test quality into a product like Fedora. You can only
attempt to assist developers in discovering issues that have escaped
their unit tests, so that through iterations of design/code/test the
package becomes stable and feature complete. It takes many iterations,
across many releases for some packages.
> 1) Updates to stable that result in any reduction of
functionality to
> the user are unacceptable.
An absolute rule containing "any" ignores reality.
That means any and all updates are unacceptable.
--
Krzysztof Halasa
The opposite of change is death. No updates as a hard and fast rule
would drive many users and packages away from Fedora.
5161
days inactive
5163
days old
106 comments
50 participants
participants (50)
-
Adam Williamson -
Al Dunsmuir -
Alexander Kahl -
Bill Nottingham -
Björn Persson -
Bruno Wolff III -
Chris Weyl -
Christof Damian -
Dan Horák -
David Malcolm -
Dominik 'Rathann' Mierzejewski -
Doug Ledford -
Ewan Mac Mahon -
Gilboa Davara -
Haïkel Guémar -
Ian Weller -
Jaroslav Reznik -
Jesse Keating -
Joe Orton -
Jon Masters -
Josh Boyer -
Josh Stone -
Julian Sikorski -
Karel Zak -
Kevin Fenzi -
Kevin Kofler -
Krzysztof Halasa -
Lennart Poettering -
Luke Macken -
Martin Langhoff -
Matej Cepl -
Mathieu Bridon -
Matthew Garrett -
Matthew Miller -
Matthias Clasen -
Michael Schwendt -
Michal Schmidt -
mike cloaked -
Rahul Sundaram -
Ralf Corsepius -
Richard W.M. Jones -
Seth Vidal -
Stephen John Smoogen -
Steven M. Parrish -
Sven Lankes -
Terry Barnaby -
Thomas Janssen -
Todd Zullinger -
Tomas Mraz -
Toshio Kuratomi