Fedora.next QA planning: generic / Product-specific release criteria and blocker review issues

[Adam Williamson]

On Wed, 2014-05-28 at 09:15 -0600, Mike Ruckman wrote:

> I think we might be able to strike a balance here between total
> separation and a unified blocker process (though it incurs a bit of
> overhead). Just as we currently mark bugs as blockers, we can continue
> to do that - but *also* mark them as F21<foo>FinalBlocker. This would
> allow for tracking at the product level but also keep some sanity with
> blocker meetings. Then, later down the road if the blocker process
> diverges with each product, they can simply drop the generic blocker
> tracking bug.
> 
> I don't immediately see much of a downside to this - but then again,
> that doesn't mean there isn't :) At the very least it might lessen the
> learning curve as we carve a new path with Fedora.next.

I thought of something similar, but just as a whiteboard field entry, to
make it less intrusive. That should be enough metadata for searches.

I should note that I was remiss in my original email in missing one
issue with the 'combined' approach: it will likely require WG
representatives to be present at most blocker meetings. I expect we'd at
least want reps of any WG whose product is obviously affected by /
involved in a given blocker to be present at the review meeting(s) for
that blocker.

For the last few releases, blocker meetings have become almost QA-only a
lot of the time, or QA plus releng; we haven't had very good
representation from devs/packagers. A 'softly softly' approach to
improving that hasn't gone very well; I think outside of QA and releng,
people have developed this mindset where the blocker review process is
something that Just Sorta Happens in the background, and they only get
involved where it touches upon them directly (e.g. they're actually
assigned a bug that's nominated as a blocker). We're probably going to
have to change that mindset in a .next world: we would already quite
*like* the expertise and input of folks outside QA/releng to the blocker
process, but in a world with Products with more domain-specific release
criteria, we're really going to need it.

> > Second, there's a similar issue of organization with regard to the
> > release criteria. I haven't explicitly written this down anywhere, but
> > I've been sort of unconsciously expecting that we'd keep the existing
> > 'generic' release criteria pages for criteria that are not
> > Product-specific, and then we'd have Product-specific release criteria
> > pages which would basically be *additive*: they'd add additional
> > criteria applicable only to that Product. They could also perhaps
> > 'patch' the generic criteria to a limited degree, though this would
> > get messy if the delta got too great.
> > 
> > However, Mike (roshi) has produced draft release criteria for the
> > Cloud product as part of his work on the 'test outline' for that
> > product -
> > https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Alpha_Release_Criteria ,
> > https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Beta_Release_Criteria ,
> > https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Final_Release_Criteria
> > - and used another possible approach; the criteria he has drawn up
> > are clearly intended to be *comprehensive* with regard to the Cloud
> > product. They would not need to be read in conjunction with the
> > 'generic' criteria.  
> 
> Just so we're clear here: I split out the cloud specific (from what I
> could tell) criteria out so that it was easier for me to keep track of
> in my head [0]. I wasn't intending it to be an end solution (even though
> I suppose it could be, at some point).

Sure, I understand that - but just the fact you did it this way
highlighted that it was actually one possible 'final' approach. Before I
saw your drafts I hadn't even really considered it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net