Review of Fedora 18 Release Criteria

[Adam Williamson]

On Tue, 2012-10-09 at 16:07 -0700, Adam Williamson wrote:
> On Tue, 2012-10-09 at 16:48 -0400, David Cantrell wrote:
> > On Tue, Oct 09, 2012 at 07:19:25AM -0600, Tim Flink wrote:
> > > As we're getting closer to the scheduled time for beta freeze, we'd
> > > like to find out now if any of the current criteria or proposed criteria
> > > changes are unreasonable to expect for beta. There may be more changes
> > > for final as we get closer to that but I think that we're pretty close
> > > to being done with the release requirements for beta.
> > > 
> > > The current (as of this writing) release criteria are available at:
> > >  - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> > >  - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> > >  - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
> 
> Thanks David! Some thoughts follow.
> 
> > I would like to see changes to the blocker criteria for each release.  The
> > first item on each release criteria is that all blockers must be CLOSED.
> > Blockers are determined by criteria defined below which always group
> > anaconda in because we cannot address those problems in a later update
> > release.  This gets us on the bug fixing treadmill as we edge closer to each
> > release because every anaconda bug more or less becomes a blocker.  
> 
> This paragraph was a bit tricky to read, but now I've given it a few
> tries, it seems to be more or less a preamble, yes? I'm not sure if
> you're suggesting that "Blockers are determined by criteria defined
> below which always group anaconda in because we cannot address those
> problems in a later update release.  This gets us on the bug fixing
> treadmill as we edge closer to each release because every anaconda bug
> more or less becomes a blocker." is a problem, or just mentioning it as
> background. It's perfectly true as background, but I don't see it as a
> problem: it's just an innate characteristic of the software you write.
> The installer is something that cannot be updated (for practical
> purposes), it must work to a high standard as shipped, because if it
> doesn't, that's a much bigger problem than a component which _can_ be
> updated not working. I agree with your assessment, but I see it as an
> inherent characteristic of an operating system installer, not any kind
> of problem in the process.

Oh, to address one point I missed - it's certainly not the case that
'every anaconda bug more or less becomes a blocker'. There have been 372
bugs reported against the 'anaconda' component with '18' as the version:
32 of them have been accepted as blockers ('AcceptedBlocker' in the
whiteboard field), that's less than 10%. 365 bugs were reported against
'anaconda' version '17', and 25 of them were accepted as blockers.

Overall these numbers seem a bit low because installation-related bugs
can wind up being in livecd-tools or pungi or lorax or dracut or several
other components, but they're sufficient to demonstrate the basic point,
nowhere near all anaconda bugs are considered blockers. I don't think
that assertion can be used to support any argument, as it seems clearly
not to be the case.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net