Criterion proposal: security

[Adam Williamson]

On Thu, 2012-10-25 at 21:57 -0600, Vincent Danen wrote:

> So whether that's done earlier in the cycle or later, it doesn't matter.
> Earlier on gives it more time to get fixed.  The end result is the same:
> it has to be fixed.  I don't see why making it a blocker in the alpha
> stage vs making it a blocker in the pre-GA stage makes a difference
> other than you're giving a developer less time to fix something that
> absolutely needs to be fixed.

Well...not really.

If a bug is an Alpha blocker it has to be fixed by Alpha.

If a bug is a Final blocker it has to be fixed by Final.

So if you start at a theoretical point one week before the Alpha gets
signed off, you have one week to fix the bug if it's an Alpha blocker.
You have, oh, about three months and one week to fix the bug if it's a
Final blocker.

I still don't understand the RHEL process entirely but it seems to
involve considering blocker status more or less independent of
milestone, so you might put 'blocker+' on a bug and have the target
release (or whatever it's called) as 'alpha', but that doesn't really
mean it's an Alpha blocker - maybe one week before the Alpha goes out,
half the bugs get the target release changed to 'beta' and the Alpha
goes out anyway.

Again, we don't do that for Fedora. We have three blocker lists per
release; Alpha blockers, Beta blockers, Final blockers. We don't take a
look at the Alpha blocker list one week before Alpha release and go
'well hmm, that looks a bit long, maybe we should bump some of them to
Beta'.

So we have weaker criteria for Alpha and Beta because, well, Alpha and
Beta release don't have to be as stable as Final releases. Given the
tight schedules of Fedora we can't just take the Final standards and
apply them to Alpha. We just don't have time. The Alpha criteria are
very minimal and define only what absolutely has to be done for an Alpha
to go out. If you look at the criteria, we can release Alphas with
gigantic flaws in them.

> >If you think it wouldn't realistic to say 'we'll take any persistent
> >moderate flaw as a blocker' that's valuable feedback...then the question
> >becomes do we only bother about important+ bugs, do we try and come up
> >with some kind of solid definition for what 'moderate' bugs we'll
> >consider blockers and which ones we won't, or do we just say 'we'll
> >evaluate 'moderate' bugs on a case-by-case basis'.
> 
> As stated above, I think you can break down the moderate flaws into
> types.  If you do that, you should rarely have to look at exceptions, I
> think.  I mean, if there is a really bad local flaw that wouldn't fit
> into the category of a what is defined as a moderate release blocker,
> but it's really that bad, then chances are it's not classified correctly
> and may actually be an important flaw.  CVSSv2 can help here.  So can
> the SRT, or the Fedora Security Response Team (many of whom are SRT
> too).  A simple "what do you think" sent to either team will get someone
> with experience to evaluate the flaw.

That sounds like a good approach, thanks.

> >Yup, that's useful feedback indeed, and thanks.
> 
> I think a policy would be extremely helpful as it takes the guess-work
> out of things.  

Definitely, that's exactly the idea.

> I'm also appreciative that you reached out for feedback
> because there is a lot of assistance that we (SRT) would _like_ to give
> to Fedora, and we're doing as much as we can, but this kind of outreach
> is really quite nice and we're more than happy to lend any kind of
> assistance that we can.
> 
> >> Conversations for another day.
> >>
> >> I would be happy with this as a first start, honestly.  A cornerstone to
> >> start building a more comprehensive policy on is nothing to be taken
> >> lightly.  =)
> >>
> >> Thanks for bringing this up, Adam.
> >
> >Thanks a lot for the input!
> 
> You're very welcome.  I hope it's helpful!  Please do let me (or the SRT
> in general) know if you need any further help, even if helping to define
> what a moderate release blocker might look like.  Again, we are more
> than happy to help.

I'll look at the points you and other responders raised and try to come
up with tweaked criteria tomorrow. Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net