Release criteria: kickstart
awilliam at redhat.com
Thu Aug 25 22:20:05 UTC 2011
On Thu, 2011-08-25 at 14:35 -0600, Stephen John Smoogen wrote:
> On Thu, Aug 25, 2011 at 14:25, Chris Lumens <clumens at redhat.com> wrote:
> >> kickstart is a very broad area; you can write extremely complex
> >> kickstart files that do a lot of stuff. So broadly what we'd need to do
> >> is define a subset of kickstart functionality that we expect to work,
> >> and then possibly divide that up by release phase (so some stuff must
> >> work by Beta, the rest by Final, for e.g.)
> >> anaconda devs following this list, do you have any existing expectations
> >> as to what level of kickstart functionality ought to be in place for
> >> releases, and when you think would be appropriate?
> >> So far it seems everyone more or less agrees that it should be possible
> >> to do at least a basic unattended kickstart install by Beta.
> > You're right, kickstart is incredibly broad. I don't think we could
> > ever hope to come up with criteria to cover all of it. I guess the best
> > we can do is define criteria in terms of something else we already have.
> I would go for the classical kickstart test for Alpha:
> Does it take a minimal kickstart and build a default system. The
> minimal being the exact stuff that would be created if a person just
> clicked through a release.
Oh, I like that. That's very good.
> For Beta
> Take these X broken kickstarts, does it bail at the appropriate places.
> Take these Y working kickstarts, does it work.
> Where Y is a set of items that can be tested on say a KVM or a
> "default" desktop defined somewhere.
I'm not so keen on that; it's a bit specific. One of my fetishes with
the criteria is to keep them generic so they don't have to keep being
changed all the time; I wouldn't want a criterion to rely on some
specific kickstart file we keep lying around somewhere.
> For Final
> The above and some subset of obscure items that can be tested reliably
> somewhere that development and QA can replicate.
again, seems a bit arbitrary; what we really need is the kickstart
functionality we actually believe has to work, not an arbitrary set of
interesting bugs we happen to be able to test.
a minimal answer to this question would be fairly valid, I guess. at
least in the short term. sometimes working up criteria in response to
real-world cases - i.e. someone finds a kickstart problem, files a bug,
proposes it as blocker, we agree it should be one and work up a
criterion in response to it - has proved a decent way of doing things in
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
More information about the test