Security release criterion proposal

[Adam Williamson]

Hey, all. The topic of whether and which security issues should block
releases has come up several times before. While we haven't actually had
many really serious security issues to worry about since the
introduction of the current release criteria system, I think it's
certainly something we should take into account. At the same time, it's
fairly established in practice that we don't just consider anything
worth issuing a CVE for as a release-blocking bug. So, to capture what I
believe would be the intent of the project, I propose this as an Alpha
release criterion for F16 onwards. (For those on devel and security
lists who may not be completely familiar with the release criteria /
blocker system, this would mean that any bug for an issue which breached
the criterion should be considered a release blocker for any Alpha, Beta
or Final release).

# There must be no known remote code execution vulnerability which could
be exploited during installation or during use of a live image shipped
with the release

Points to consider:

* Possible variants to the type of vulnerability covered...do we also
want to make local privesc vulns blocking? Conversely, do we want to
make only remote *root* execution vulns blocking? I don't know if anyone
would want to go as far as making DoS vulns release blocking, but speak
up if you would! (Of course there is again the local/remote distinction
to consider there: 'all DoS vulns' would be a much tighter standard than
'remote DoS vulns').

* The caveat about where the issue is exploitable is important because
if you can't exploit it from the installer or a live image, it becomes
less relevant whether we ship with it or not, because you would be safe
with the first post-install update assuming we submitted an update for
the issue promptly. But it may be the case that we want to broaden it
out to also cover issues that can be exploited from a default DVD
install, if we consider the window between install and first update (if
updates repos aren't used during installation) to be unacceptable.

* We have a system whereby the criteria get more onerous from Alpha to
Beta to Final, so it could be the case that we accept worse security
issues in Alpha than in Beta and so on; we could have a loose criterion
for Alpha and a tighter one for Beta. In this case it felt sensible to
me to just go with one criterion, but please say if you disagree.

* The aim of the release criteria is more to codify existing practice
than to extend it; we have taken security issues as release blockers
before and considered but rejected less serious ones, but we have done
so on an ad hoc basis. I tried to write the criterion to reflect our
past practice on this. So precedents are important here; if you have any
that contradict the proposed criterion, please do cite them.

Feedback please! Thanks :)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net