Security release criterion proposal

[Kevin Kofler]

Adam Jackson wrote:

> On 5/18/11 4:49 PM, Kevin Kofler wrote:
>> The thing is, if we block the release for each and every known security
>> issue, considering the time passing between notification and public
>> availability of a fix, we will never be able to release anything. We have
>> to draw the line somewhere, and the best way to do it is to use our
>> time-based schedule.
> 
> False induction.

Uh no, you can't dismiss my argument that easily…

If, say, it takes 2 weeks to get a fix for a critical security issue 
published/publishable (i.e. developed and put out of embargo), and if 
there's at least 1 critical security vulnerability in every 2-week interval, 
we will mathematically never be able to release if we treat all critical 
security vulnerabilities as blockers. (And that sentence remains true no 
matter how you define "critical".) And this is just one of the possible 
constellations, and a very simplified one to prove my point. In practice, 
things are much more stochastic, but there are many possible scenarios in 
which each slippage will cause another.

You show no evidence whatsoever showing that this is not going to happen. So 
this means you MUST consider the scenario I pointed out at least as a worst-
case scenario. (Empirically, I'd actually expect it to be the average-case 
scenario, but I'll admit I have no statistical evidence to support that.)

> Now you're drawing lines.  Before you were saying "this is impossible,
> we shouldn't try".  Moving the goalposts.

Call me when you see a security issue which actually fits my description 
(i.e. an automated worm which successfully exploits a service that's enabled 
by default, listening to more than just localhost by default and not 
firewalled by default), and that during the short release candidate time 
window, even. I haven't seen it happen in any of the 14 releases we did nor 
the 15th we're doing now, and I don't see it happening any time soon. This 
is purely theoretical.

Now I won't object to putting this exact item on the blocker list, but I 
don't expect it to get invoked, ever. I DO object to putting anything 
security-related OTHER than that up as a blocker, because we need to release 
at some point.

        Kevin Kofler