2009-06-17 - Fedora QA meeting Agenda

[James Laska]

Greetings,

Comments below ...

On Tue, 2009-06-16 at 16:58 -0700, John Poelstra wrote:
> James Laska said the following on 06/16/2009 12:23 PM Pacific Time:
> > Fedora QA Meeting
> > Date: 2009-06-17
> > Time: 16:00 UTC (12 EDT)
> > Location: #fedora-meeting on irc.freenode.net
> >
> >          1. Previous meeting follow-up ...
> >          https://fedoraproject.org/wiki/QA/Meetings/20090610
> >
> >          2. F-11 retrospective feedback
> >
> >          3. QA feedback of related FAD topics
> >
> >          4. AutoQA - the wheels are in motion
> >
> 
> I have a conflict and will not be able to attend tomorrow.
> 
> 5. There have been a few different conversations about adding more 
> granularity to the schedule, more blocker bug days, and making the 
> hand-off between releng and QA clearer.
> 
> I've taken a shot at doing that here:
> http://poelstra.fedorapeople.org/schedules/f-12/f-12-releng-tasks.html
> (right now releng is the most detailed and complete... I'll create other 
> team schedules for QA, etc. after we get the kinks out of this one.)
> 
> Key take-aways:
> 
> 1) Blocker bug reviews happen on Fridays before key freezes

I'm all for spelling these out ahead of time.  

> 2) Exact date of releng composes are specified... hopefully this makes 
> it easier to plan ahead.
> 3) Exact dates of testing of releng composes by QA are specified

I know it's hard to predict when content will be ready for composes, but
adding some level of predictability to the release schedule really helps
QA plan ahead for organizing test efforts.  Perhaps we can tie this into
the contingency section below and identify how much wiggle room is (or
isn't) available when it comes to composes.

> 4) Composes are scheduled the day following a freeze (tuesday). 
> Naturally there can be multiple composes, but I am guessing that with 
> limited resource, QA can only commit to testing certain ones vs. every 
> compose that shows up.  I think this gets at the points raised, but if 
> it doesn't let me know and I'll put the dates and tasks on the schedule 
> a different way.

Yeah, Jesse and I were discussing this during the Fedora 11 release
candidate phase.  Focusing on planned composes, and documenting what
testing can be performed against rawhide, helps QA reduce the [re-]start
cost of downloading release candidate ISO kits for each new compose.  So
I'm in favor of trying this approach for the next release.

> 5) For all releases, including GA, there are two composes with time 
> scheduled to test them.

Cool, same as above.  There was some discussion during Fedora 11 about
adding these compose checkpoints.  I like having them explicit like
this.

> Open Questions:
> 1) What tweaks should I make to better reflect QA's needs?

We talked in the past about adding Test Days to the formal QA calendar.
While it would probably make things better in terms of one stop schedule
shopping, I'm happy with continuing to modify the Test Day schedule
(which changes frequently) in the wiki.  Unless you have any
thoughts/recommendations?

> 2) What tweaks should I make to better reflect releng's needs (I'm 
> assuming Jesse will be there).
> 3) What contingencies should we document and intact if composes are 
> late, testing isn't finished, or tests fail?

I like this idea, although I don't have strong experience to provide any
examples of what has worked or not worked in the past.  Hopefully not
too much of a pipe dream, but some sort of contingency gameplan would be
an interesting idea.  

First, it would communicate to other interested parties how the release
team intends to address the "oh rats!" moments of software development.
Second, this might serve to address a theme of the recent FAD around
documenting standard operating procedures that the release team follows.
That said, I'm not sure what it should look like.  Perhaps we look at
what we did during the F11 slips and add formalize some structure around
that.

> Now that we have matured and have a larger QA presence I think we should 
> move to more of a standard software model (just as we did with the 
> naming of the schedule, etc.) where there is more separation between 
> Releng and QA.  IOW, Releng provides the service of packing the bits and 
> composing an installable distro and QA provides the service of testing 
> them and giving a thumbs up/down on them.

Thanks,
James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20090617/200795bf/attachment.bin