Fedora.next QA planning: generic / Product-specific release criteria and blocker review issues

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/27/2014 09:22 PM, Adam Williamson wrote:
> Hi, folks. So continuing with the theme of setting up the
> Fedora.next QA process, I thought before going too far with draft
> release criteria etc, we could discuss a couple of important points
> that have come up since I sent out the draft test plan.
> 
> There are two kind of similar issues in particular I'm thinking of.
> 
> 
> First, at the QA meeting this week, tflink pointed out that we
> would need to decide whether we will have sort of 'parallel'
> blocker/freeze exception bug processes for each product. That is,
> do we have:
> 
> F21FinalBlocker F21ServerFinalBlocker F21WorkstationFinalBlocker
> 
> etc etc, and separate lists of blockers on 
> http://qa.fedoraproject.org/blockerbugs/current per product (and 
> non-Product-specific), and separate meetings? Or do we keep the 
> 'unified' blocker nomination / review process, and just handle
> blocker bugs for all Products within it?
> 
> At first blush I'd incline to keeping a single unified process,
> because splitting them up seems like an awful lot of work and I'm
> not sure it comes with a clear benefit. It also relies either on
> reporters correctly determining what product their bug is relevant
> to, or on a triage step for blocker nominations.
> 
> However, it's worth noting that if we allow the release schedules
> for the Products to diverge in future, it would probably then be
> inevitable that we'd need split blocker review (and release
> validation, for that matter).
> 

I'm of the opinion that we should keep the blocker process unified
until and unless that becomes impossible. The only benefit I can think
of to separating them would be metrics, and I cannot measure the
degree to which I don't care about metrics :)


> Second, there's a similar issue of organization with regard to the 
> release criteria. I haven't explicitly written this down anywhere,
> but I've been sort of unconsciously expecting that we'd keep the
> existing 'generic' release criteria pages for criteria that are
> not Product-specific, and then we'd have Product-specific release
> criteria pages which would basically be *additive*: they'd add
> additional criteria applicable only to that Product. They could
> also perhaps 'patch' the generic criteria to a limited degree,
> though this would get messy if the delta got too great.
> 
> However, Mike (roshi) has produced draft release criteria for the
> Cloud product as part of his work on the 'test outline' for that
> product -
> 

https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Alpha_Release_Criteria
,
https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Beta_Release_Criteria
,
https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Final_Release_Criteria
- - and used another possible approach; the criteria he has drawn up are
clearly intended to be *comprehensive* with regard to the Cloud product.
They would not need to be read in conjunction with the 'generic' criteria.
> 
> I'm not as sure which approach I prefer here, and to a degree the 
> difference between them isn't as clear cut as it appears at first 
> glance; however the criteria are ultimately presented, we'll likely
> have several that are applicable to multiple Products. Even if
> these are displayed multiple times on 'comprehensive' pages for
> each Product, they might be shared at the 'source' level using
> mediawiki transclusion, for instance (to prevent them diverging
> unintentionally). And of course we aren't necessarily tied to
> mediawiki for presenting the criteria, which is another factor to
> bear in mind (I know one of tflink's Grand Plans involves a
> different way of maintaining and presenting the release criteria).
> 
> Thoughts on the best approach for each of these issues would
> certainly be appreciated! I thought it'd be best to take some time
> to think them over before moving ahead with drafts and so on.
> 


I like the transclusion idea, personally. We can have individual pages
for each Product and follow them straight through (this also reduces
the amount of paging about and information to keep track of).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlOF0mQACgkQeiVVYja6o6NzCwCcDHsuB2+EEHqrFrVcv8E7t3CQ
ec4AoJha8SuPxOde0SKpLBH8rUMIKPve
=B81N
-----END PGP SIGNATURE-----