[Fedora-packaging] critical path security update policy

[Michael Schwendt]

On Mon, 20 Apr 2015 11:03:44 +0200, Mario Torre wrote:

> Hmm,
> 
> Security has precedence over even backward compatibility.

Said that and applied a _tested_ Yum update (a few positive votes in
bodhi) that resulted in Yum not working at all anymore. Worst-case
scenario for all users that rely on Yum for updating their installations.
It has been discussed quite a lot after it had happened. It's just an
example here. (WDYT why three installed kernels are kept?)

Once package maintainers have made the experience, they (hopefully) become
more careful, and that is one reason why some updates require positive
feedback from more testers than just 1-3. Responsible maintainers then
require positive feedback from many more testers.

> The maintainers should be ultimately responsible to ensure that the
> package they maintain is in a coherent state and in theory just
> backporting the security patches. I know that is is often easier said
> than done, but the general rule is security first.

Cheap talk. ;)

I've mentioned already that even if it were a patch only, it would still
need a complete new build of the package with build dependencies that may
have changed meanwhile in ways you don't imagine.

It has happened before that even a "simple rebuild" after just a few
modifications in the package spec file, resulted in broken software.
People say "the packager should notice". What if he/she isn't affected,
and a single tester perhaps isn't either, but many other users are?

> General users can't really be asked to enable by default a testing
> repository, and you really need to know if an update is a security
> update, rather than a general update.

Face the facts!

Many users do install test-updates *unknowingly*, because they install the
same packages that have been offered in updates-testing repo for the
minimum time _without_ any positive tester feedback and with the update
submitter triggering the push to stable manually.

These users become consumers of updates-testing as soon as the same
updates are offered in a repo with a different name that's enabled on
their machines. And it isn't bad as long as bugs, which are found, get
fixed. An update a day keeps the doctor away - but it is so much more
clever to find'n'fix some bugs while an update is still in updates-testing.
If people wait for the same updates to appear as "stable updates", that
only delays the testing. Not wise.

It's worse, if a fix takes time or is complicated because of unforeseen
problems, and if it's an update that cannot be withdrawn anymore because
it has entered the stable updates repo already.

Better if bugs are avoided/fixed prior to release, but (face the facts)
consider all the cases where upstream developers haven't noticed, where
package maintainers are not aware of issues, and where testers have not
tested all features. Many more bugs are found after release.  *After*
upstream has made a new release. *After* Fedora has offered builds in
stable updates repo, and not in updates-testing already.

There are other examples. Updates to build tools: the user's current
project files become incompatible, even if only in small ways. The user's
project work is interrupted. Hell breaks loose, any the user will run wild
- publicly because such incidents likely turn into pet peeves. Updates to
games: everything works great, just the user's savegames expose a
regression, and the user doesn't notice it immediately.
The user *could* have taken precautions, such as making backups before
applying the updates, but why be afraid of "stable updates"? If the user
had taken an earlier peek at updates-testing, maybe he would have acted
differently. In case a test-update really is faulty, it is so much easier
to roll back than when the same update is offered in stable updates repo.

Part of the problem is that users expect a much higher quality from
packages in the "updates" repo than from _the same_ packages when they are
still in the "updates-testing" repo.
Once the same stuff enters the stable updates repo, they don't spend any
time in the Fedora Update System checking for test results. They run
commands, such as "yum -y update", and expect everything to work. If
they knew how many of the updates they install that way are with 0 karma
points in the updates system, maybe they would reconsider. So, they learn
it the hard way. Run into trouble with stable updates a few times, and
slowly learn that they have been using untested test-updates all the time.
Same for version upgrade requests submitted by users in bugzilla. The
updates appear in updates-testing, nobody gives them a try (because
consumers fear instabilities and don't want to install everything from
updates-testing), so the testing starts only as soon as packager has
pushed to stable, and then everybody gets the same updates. This makes no
sense. Users have the power to fix it. Keep an eye on the software you're
interested in. Earlier than when it appears in "updates" repo.