The new Update Acceptance Criteria are broken

[Jon Masters]

On Tue, 2010-11-16 at 23:35 +0100, François Cami wrote:
> On Fri, Nov 12, 2010 at 9:14 PM, Kevin Fenzi <kevin at scrye.com> wrote:
> > On Fri, 12 Nov 2010 14:54:28 -0500
> > Simo Sorce <ssorce at redhat.com> wrote:
> >
> >> Adam why should security updates wait at all ?
> >> Do you fear some packager will flag as security updates that are not ?
> >> Surely we can deal with such maintainer if that happens...
> >
> > No. The issue is that in the past sometimes security updates have been
> > rushed out with no testing and broken things badly. ;(
> >
> > See http://fedoraproject.org/wiki/Updates_Lessons
> > For some small number of examples (yes, anyone is welcome to please add
> > others you have run into to the page).
> >
> > I know of at least dbus, bind, nss and a few others that were security
> > updates and pushe out with no testing and turned out to break things.
> >
> > Perhaps security updates could have a smaller timeout?
> 
> This reminds me of the excellent "Timing the application of security
> patches for optimal uptime" paper:
> http://www.homeport.org/~adam/time-to-patch-usenix-lisa02.pdf
> Maybe a smaller timeout would do, yet I don't think that we could use
> a small enough (security-wise) timeout and still be safe from awful
> regressions.

Funnily enough, that paper concludes that 10 days is an optimal time to
delay patch application in order to avoid getting a bad patch that will
need later revision. This is close to the 7 days used for pre-release
updates in the Updates Policy. So I would take this as another useful
data point demonstrating that one doesn't need to push security updates
instantaneously to stable, but that they can wait for testing results.

(no policy is immune from huge day zero vulnerabilities with massive
exploits, but I'm certain that if there were such an incident with this
policy, an exception could be made if the impact was significant)

Jon.