Release criteria: kickstart

[Stephen John Smoogen]

On Thu, Aug 25, 2011 at 14:25, Chris Lumens <clumens at redhat.com> wrote:
>> kickstart is a very broad area; you can write extremely complex
>> kickstart files that do a lot of stuff. So broadly what we'd need to do
>> is define a subset of kickstart functionality that we expect to work,
>> and then possibly divide that up by release phase (so some stuff must
>> work by Beta, the rest by Final, for e.g.)
>>
>> anaconda devs following this list, do you have any existing expectations
>> as to what level of kickstart functionality ought to be in place for
>> releases, and when you think would be appropriate?
>>
>> So far it seems everyone more or less agrees that it should be possible
>> to do at least a basic unattended kickstart install by Beta.
>
> You're right, kickstart is incredibly broad.  I don't think we could
> ever hope to come up with criteria to cover all of it.  I guess the best
> we can do is define criteria in terms of something else we already have.

I would go for the classical kickstart test for Alpha:

Does it take a minimal kickstart and build a default system. The
minimal being the exact stuff that would be created if a person just
clicked through a release.

For Beta

Take these X broken kickstarts, does it bail at the appropriate places.
Take these Y working kickstarts, does it work.
Where Y is a set of items that can be tested on say a KVM or a
"default" desktop defined somewhere.

For Final
The above and some subset of obscure items that can be tested reliably
somewhere that development and QA can replicate.



-- 
Stephen J Smoogen.
"The core skill of innovators is error recovery, not failure avoidance."
Randy Nelson, President of Pixar University.
"Let us be kind, one to another, for most of us are fighting a hard
battle." -- Ian MacLaren