Criterion proposal: security

[Adam Williamson]

On Wed, 2012-10-24 at 20:10 -0600, Vincent Danen wrote:

> >What's your opinion on "moderate bugs that can't be fixed by
> >updates" (i.e. persistent flaws) as a Final blocker? Is that going too
> >far?
> 
> I don't think that's going too far, but again it should be evaluated on
> a case-by-case basis.  Some moderates are more serious than others.  We
> often fix moderate flaws in errata, but there are plenty that we defer.
> They are, by classification, moderate but just aren't worth the effort
> to do a fix for it "right now".  I think the same could be done here.
> Important+: no question.  Moderate: maybe it needs to get some acks
> before it is considered a blocker?

What we're trying to do with the criteria is reduce the occurrence of
'case-by-case evaluation' as much as we can. It's inevitably the case
that we can't entirely eliminate it, but we want to make things as
objective as we can.

> For instance, a moderate flaw that only would affect a small subset of
> users probably shouldn't be a release blocker if it's found a week
> before GA.  One that affects 90% of users probably should.
> 
> Again, I don't know the process here, but maybe it would make sense to
> have moderate flaws block the alpha (and maybe the beta?) but not later.
> This gives you an out -- two weeks to release and someone finds a
> moderate flaw, you don't have to necessarily delay the release.  But if
> it's found a month before GA and can be fixed prior to the alpha
> release, maybe that should be considered just so that it gets done.  (My
> fear otherwise is that if you have critical-only as blocking alpha, you
> _have_ to leave it to a later stage that might be more inconvenient).

Ah, the old RHEL attitude again ;) Funny how things come in twos. This
seems to be a RHEL strategy - throw everything but the kitchen sink on
the 'blocker' list early in the process, and get tighter closer to
release.

That's not how we do things in Fedora, we do things the other way
around, really. Note that in Fedora the blocker list is not a todo list
or a 'let's not forget about these bugs' list. It is a _list of bugs
that block the release_. That's all it is. No bug is ever intended to be
on the list of blockers for a given release unless it's really and truly
blocking that release. We don't use the blocker list for 'just so that
it gets done' purposes. We decided a few years back that that way of
doing things kind of sucked, and designed a more rigorous way of doing
things :)

If you think it wouldn't realistic to say 'we'll take any persistent
moderate flaw as a blocker' that's valuable feedback...then the question
becomes do we only bother about important+ bugs, do we try and come up
with some kind of solid definition for what 'moderate' bugs we'll
consider blockers and which ones we won't, or do we just say 'we'll
evaluate 'moderate' bugs on a case-by-case basis'.

> >Yeah, that would be too ambitious. The other difference in Fedora is we
> >don't consider 0-day updates part of a release. We ship a bunch of
> >0-days, but they're just not considered in the 'release process', except
> >in very special circumstances. Blocker bugs have to be fixed *in the
> >release package set* (what gets frozen).
> 
> That makes sense.  I didn't think that Fedora had "0-day errata" like we
> do with RHEL releases.  They just get pumped out on or after release
> date, regardless of security-relevance (at least that's what it seems to
> me, but as far as Fedora is concerned, I'm just an end-user).

The way you say it it does sound like there's a difference, because for
Fedora, a 0-day update is just any update that happens to have been
pushed stable by the official release date. There's no special handling
of such updates, they just happen. What actually happens is that we lift
freeze after the go/no-go meeting but before release; any update that
accumulates sufficient karma and gets pushed 'stable' in the period
between the freeze being lifted and the release going out becomes a
0-day update, i.e., it's available on release day. There's no special
shepherding of such updates, usually.

> Absolutely.  I didn't mean to imply you shouldn't make it a policy
> because it's more often than not a non-issue.  Rather, I meant it should
> be easy enough to get into place as a policy because it's been easily
> achieved (so far), so can't really be seen as a major undertaking or
> "too much to ask for" kind of thing.

Yup, that's useful feedback indeed, and thanks.

> Conversations for another day.
> 
> I would be happy with this as a first start, honestly.  A cornerstone to
> start building a more comprehensive policy on is nothing to be taken
> lightly.  =)
> 
> Thanks for bringing this up, Adam.

Thanks a lot for the input!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net