F17 QA schedule modification proposal

[Adam Williamson]

On Mon, 2011-12-05 at 11:34 -0700, Robyn Bergeron wrote:
> On 11/29/2011 04:01 PM, Adam Williamson wrote:
> > Hi, Robyn. As discussed on the test list this week and at our meeting,
> > we'd like to propose the schedule modification described in
> > https://lists.fedoraproject.org/pipermail/test/2011-November/104583.html
> > to the F17 QA schedule. Could you put that in there? Thanks!
> Sorry - this mail got filtered in ways that are all sorts of wrong. :(
> 
> My only concern with the schedule as proposed is that we are essentially 
> doing the Alpha Test Compose prior to Feature Freeze, which is the point 
> where features should be testable. I've popped in the existing dates 
> below, but I just want to point out that I think we know from past 
> experience that a lot of features don't hit their feature freeze 
> deadline until *on the actual deadline*, which means that we are 
> essentially testing a compose of stuff a full week before I expect 
> people to finish procrastinating and getting their stuff done.  We still 
> have the actual release candidate after that, but I don't want us to 
> basically pull in a date and find out that it's not helpful because all 
> the breaking pieces get pulled in/finished after the test compose date.
> 
> Pre-Alpha Rawhide Acceptance Test Plan #1    2012-01-19
> Feature Submission Deadline		     2012-01-24	
> Pre-Alpha Rawhide Acceptance Test Plan #2    2012-01-26
> Test Alpha 'Test Compose'                    2012-01-31
> Feature Freeze (Testable|Complete)	     2012-02-07
> Fedora 17 Alpha Go/No-Go Meeting             2012-02-22
> 
> Test Beta 'Test Compose'                     2012-03-06
> Fedora 17 Beta Go/No-Go Meeting              2012-03-28
> 
> Test 'Final' Test Compose (TC)               2012-04-09
> Fedora 17 Final Go/No-Go Meeting             2012-05-01
> 
> 
> Or it's possible that I'm nuts and it's not worth worrying over.  :)  We 
> do the Test Compose for Beta a week before Features are required to be 
> at 100%, but of course, they should already be testable/complete at 
> alpha, so the breakage scenario in Beta is far smaller, theoretically.

I think in practice it's not such a huge deal. As far as release
validation goes we just don't really *care* about the feature process
very much.

The main thing a TC achieves for us is to give us a sanity check on the
compose process - can we actually compose a full set of images at this
time, are they crazily over-size, etc - and the ability to start
identifying blocker bugs. I just don't see that it's actually a problem
if the TCs contain features that are incomplete, or if some feature
lands in TC3 that wasn't in TC1. All the work we did identifying
blockers in TC1 is still likely to be of value, and even if TC3
introduces some new blocker, we haven't *lost* anything.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net