Drawing lessons from fatal SELinux bug #1054350

[Michael Schwendt]

On Fri, 24 Jan 2014 12:39:55 -0800, Adam Williamson wrote:

> > https://fedoraproject.org/wiki/Critical_path_package#Actions
> > 
> >  | Packages within the critical path are required to perform the
> >  | most fundamental actions on a system. Those actions include: 
> >  |
> >  | [...]
> >  | get updates 
> >  | [...]
> > 
> > 
> > How to understand that?
> 
> It also says:
> 
>      graphical network install
>     post-install booting
>     decrypt encrypted filesystems
>     graphics
>     login
>     networking
>     get updates
>     minimal buildroot
>     compose new trees
>     compose live 
> 
> Are we to be expected to re-test every single one of those actions with
> every single critical path update? That seems unreasonable. If that were
> the approach, I think Kevin would have an actual apoplexy while waiting
> for his updates to get released. :)

It would be unreasonable for a single tester, but "the more testers, the
better". 

Which doesn't mean the group of testers will perform _all_ of the
(re-)tests. It means: the more testers get a chance to apply Test Updates
*and* continue using their systems normally for a few days, the more
likely it could get that some will notice issues at run-time, boot-time,
compile-time, to mention a few.

That's why I think there's reason to be very careful and sometimes even
prefer a +0 (with a comment) over a very early over-ambitious +1.

And guess what happens in non-critpath updates after 7 days and _no_
feedback. Packagers push the update manually. Sometimes with broken
deps. Sometimes the testing starts no sooner than when the update arrives
in the stable updates repo and the first real user becomes the "guinea
pig".

> > Is it so hard for testers to slow down a bit until such a system will be
> > available? ;-)
> 
> Well, at present, it kind of is, because we have this Bodhi-Bugzilla
> interaction where when an update is submitted that claims to fix a given
> bug, Bodhi posts a comment on the bug that says 'try this update and
> leave positive karma if it fixes your bug'.

Good point. Raises the question why an update that links so many bugzilla
tickets can be marked stable automatically after a +3, which may be even
about a single bz ticket.

> I mean, we could tweak that process, but I still think the Grand Bodhi
> 2.0 Vision is much more interesting, because if we have multiple karma
> types we can still take and use fast 'this fixed my bug' feedback
> however we feel it to be appropriate, while having _much_ more
> flexibility to do 'valuation' of feedback on a 'type of feedback' and
> 'package being updated' basis. Again, hate to sound like a broken
> record, but it's just hard to get enthusiastic about trying to twiddle
> the edges of the process when the process is fundamentally inadequate.

That's understood, of course.