plan for merging clumens branch onto master

[Chris Lumens]

Hey everyone, as you may have noticed I have been working on tests for
anaconda's storage code.  With jlaska's help, I've got it in shape now
where it runs just like I want.  It does all the testing inside a VM and
then communicates results to the outide world.  My existing tests should
cover the whole partitioning part of the test matrix, aside from
resizing since we can't express that with kickstart.

All of the code is on the clumens branch in tests/anaconda_storage/.
Eventually, I'd like to move it into tests/anaconda/storage/ instead
since we are adding some different anaconda tests on the branch as well.

Anyway I would like to get this stuff merged to master soon (after I
reorganize though) so it can be running and reporting results.  What do
I need to do to get this on the plan?

- Chris

[James Laska]

On Tue, 2010-11-30 at 13:54 -0500, Chris Lumens wrote:
>  Hey everyone, as you may have noticed I have been working on tests
> for
>  anaconda's storage code.  With jlaska's help, I've got it in shape now
>  where it runs just like I want.  It does all the testing inside a VM and
>  then communicates results to the outide world.  My existing tests should
>  cover the whole partitioning part of the test matrix, aside from
>  resizing since we can't express that with kickstart.
>  
>  All of the code is on the clumens branch in tests/anaconda_storage/.
>  Eventually, I'd like to move it into tests/anaconda/storage/ instead
>  since we are adding some different anaconda tests on the branch as well. 
Agreed.  Chris and I talked about this in a bit more detail.
Medium-term, I think the 'autoqa' script will need to adjust how it
locates applicable tests.  Currently, it inspects a flat directory
(os.listdir(conf['testdir'])]).  This works great for the small number
of tests we currently manage.

One way to accommodate nested test directories, would be to add support
for multiple directories to the 'testdir' autoqa.conf option.  By
itself, I don't think that'll be enough.  I think we may want to adjust
'autoqa' to use os.path.walk() when looking for tests.  Additionally, to
handle maintainers writing tests in their git-dist checkout, we'll
likely want to provide multiple values for 'testdir' to accept multiple
directories (autoqa tests and git-dist tests).

>  Anyway I would like to get this stuff merged to master soon (after I
>  reorganize though) so it can be running and reporting results.  What do
>  I need to do to get this on the plan? 
We'll need to get this on https://fedoraproject.org/wiki/AutoQA_Roadmap.
Since all of the changes in the 'clumens' branch are isolated to new
tests only, the risk of breaking other tests or core functionality is
low.  Kamil, do you think this is appropriate to include in 0.4.4?

Chris: Do we want to get the tests in and running first, and then adjust
'autoqa' to be more flexible in locating tests (e.g. honoring tests in
sub-dirs like tests/anaconda/{storage,compose,checkbot,...})?  I'll
probably want to convert our mini anaconda autoqa roadmap [1] into TRAC
tickets so we can align this with the releases better.

Thanks,
James

[1]
https://fedoraproject.org/wiki/User:Jlaska/Fedora_15_Installation_automat...

[Chris Lumens]

>  Chris: Do we want to get the tests in and running first, and then
> adjust
>  'autoqa' to be more flexible in locating tests (e.g. honoring tests in
>  sub-dirs like tests/anaconda/{storage,compose,checkbot,...})?  I'll
>  probably want to convert our mini anaconda autoqa roadmap [1] into TRAC
>  tickets so we can align this with the releases better. 
Personally, I'd get the subdir stuff working first and then get these
new anaconda tests merged over.  I just think it's kind of silly to
merge something onto master when you know there's going to be something
else major to do afterwards.  In this case, we know that all the
anaconda subdirectories would have to get reorganized.

- Chris