Re: plan for merging clumens branch onto master

[Kamil Paral]

----- "Chris Lumens" <clumens(a)redhat.com&gt; wrote:

>  Hey everyone, as you may have noticed I have been working on tests
>  for
>  anaconda's storage code.  With jlaska's help, I've got it in shape
>  now
>  where it runs just like I want.  It does all the testing inside a VM
>  and
>  then communicates results to the outide world.  My existing tests
>  should
>  cover the whole partitioning part of the test matrix, aside from
>  resizing since we can't express that with kickstart.
>  
>  All of the code is on the clumens branch in tests/anaconda_storage/. 
Wow, very complex test. I won't of course study all the code in a large
detail, but I'll provide some feedback for the test object (and other 
AutoQA-specific files) soon.

>  Eventually, I'd like to move it into tests/anaconda/storage/
> instead
>  since we are adding some different anaconda tests on the branch as
>  well. 
We can do such changes, but I can't guarantee it will be included in
the next release. But, your test works independently of those changes,
right? If you prefer, we can include your test in the next release
and change the path slightly later. I don't really see a problem in that.

>  
>  Anyway I would like to get this stuff merged to master soon (after I
>  reorganize though) so it can be running and reporting results.  What
>  do
>  I need to do to get this on the plan? 
I think you just did it :) If we can cleanly include it in our current
code (it seems we can), there's no reason why it should not be part of
the next release (0.4.4). Unfortunately I don't have a spare bare metal
machine to test your code. (I should inquire how to use RHTS). How long
does the test run take? Are there any issues we should be aware of?

Thanks,
Kamil

plan for merging clumens branch onto master

[James Laska]

On Thu, 2010-12-02 at 11:50 -0500, Kamil Paral wrote:
>  ----- "Chris Lumens" <clumens(a)redhat.com&gt; wrote:
>  
>  > Hey everyone, as you may have noticed I have been working on tests
>  > for
>  > anaconda's storage code.  With jlaska's help, I've got it in shape
>  > now
>  > where it runs just like I want.  It does all the testing inside a VM
>  > and
>  > then communicates results to the outide world.  My existing tests
>  > should
>  > cover the whole partitioning part of the test matrix, aside from
>  > resizing since we can't express that with kickstart.
>  > 
>  > All of the code is on the clumens branch in tests/anaconda_storage/.
>  
>  Wow, very complex test. I won't of course study all the code in a large
>  detail, but I'll provide some feedback for the test object (and other 
>  AutoQA-specific files) soon. 
Yeah, if we had support for it ... this would be the type of test that
would live in the maintainers dist-git directory.  But, since
anaconda/installer is such an important piece of software, I thought it
would be good for us to move forward to with integrating existing
scripts into autoqa for the benefit of F15.

>  > Eventually, I'd like to move it into tests/anaconda/storage/
> instead
>  > since we are adding some different anaconda tests on the branch as
>  > well.
>  
>  We can do such changes, but I can't guarantee it will be included in
>  the next release. But, your test works independently of those changes,
>  right? If you prefer, we can include your test in the next release
>  and change the path slightly later. I don't really see a problem in that. 
I've never done git-mv before ... I don't know if that's a painful
process?  If I have a spare moment, I'll see how invasive it will be to
update 'autoqa' to improve test discovery.
 
>  > Anyway I would like to get this stuff merged to master soon
> (after I
>  > reorganize though) so it can be running and reporting results.  What
>  > do
>  > I need to do to get this on the plan?
>  
>  I think you just did it :) If we can cleanly include it in our current
>  code (it seems we can), there's no reason why it should not be part of
>  the next release (0.4.4). Unfortunately I don't have a spare bare metal
>  machine to test your code. (I should inquire how to use RHTS). How long
>  does the test run take? Are there any issues we should be aware of? 
I can loan you my spare bare metal system I've been using to help Chris
move this stuff forward.  We can exchange access details on IRC.

On my spare system, which is *much* less beefy than what we have in the
autoqa deployment ...
      * anaconda_storage takes ~10-15 minutes to run
      * anaconda_checkbot takes ~3-5 minutes to run

I've got the wrapper (control*) setup and working well using 'autoqa
--local'.  Both tests are designed to execute on 'post-tree-compose',
and anaconda_checkbot is designed to run on 'post-koji-build' [1]

Thanks,
James

[1] Long term ... this test would be triggered by some form of
post-git-update watcher

plan for merging clumens branch onto master

[Chris Lumens]

>  Yeah, if we had support for it ... this would be the type of test
> that
>  would live in the maintainers dist-git directory.  But, since
>  anaconda/installer is such an important piece of software, I thought it
>  would be good for us to move forward to with integrating existing
>  scripts into autoqa for the benefit of F15. 
Having this stuff in anaconda git would be ideal, I think, but for now I
am happy with it living in anyone's git so long as it runs.

>  I've never done git-mv before ... I don't know if that's
> a painful
>  process?  If I have a spare moment, I'll see how invasive it will be to
>  update 'autoqa' to improve test discovery. 
It's easy.  The only thing you need to consider is that you may need to
add --follow to "git blame" and "git log" to make sure you see
entries
from before the move.

>  I can loan you my spare bare metal system I've been using to help
> Chris
>  move this stuff forward.  We can exchange access details on IRC.
>  
>  On my spare system, which is *much* less beefy than what we have in the
>  autoqa deployment ...
>        * anaconda_storage takes ~10-15 minutes to run
>        * anaconda_checkbot takes ~3-5 minutes to run 
Either you have a way faster machine than I do, or you are not including
livecd creation time in the running time.  Since that'd have to happen
on autoqa runs, it should probably be included in the overall total.

- Chris

plan for merging clumens branch onto master

[James Laska]

On Thu, 2010-12-02 at 14:03 -0500, Chris Lumens wrote:
>  > On my spare system, which is *much* less beefy than what we have
> in the
>  > autoqa deployment ...
>  >       * anaconda_storage takes ~10-15 minutes to run
>  >       * anaconda_checkbot takes ~3-5 minutes to run
>  
>  Either you have a way faster machine than I do, or you are not including
>  livecd creation time in the running time.  Since that'd have to happen
>  on autoqa runs, it should probably be included in the overall total. 
This will teach me to run `time` before giving estimates .. or multiply
my 'gut' metrics by 3. :)  

I suspect the performance on our production systems will be better, but
on a Intel(R) Xeon(R) X3440 [1] with 2G of memory ...

= anaconda_storage =
real    43m33.039s
user    11m46.798s
sys     3m22.266s

= anaconda_checkbot =
real    12m48.407s
user    5m24.264s
sys     0m50.297s

Thanks,
James

[1] http://ark.intel.com/Product.aspx?id=42928

plan for merging clumens branch onto master

[Chris Lumens]

>  This will teach me to run `time` before giving estimates .. or
> multiply
>  my 'gut' metrics by 3. :)  
>  
>  I suspect the performance on our production systems will be better, but
>  on a Intel(R) Xeon(R) X3440 [1] with 2G of memory ...
>  
>  = anaconda_storage =
>  real    43m33.039s
>  user    11m46.798s
>  sys     3m22.266s 
I just pushed a patch that reduces runtime to:

1017.85user 459.16system 31:23.37elapsed 78%CPU (0avgtext+0avgdata 2168368maxresident)k

It'll be hard to make this much faster, though.

- Chris

plan for merging clumens branch onto master

[Chris Lumens]

>  Wow, very complex test. 
Yeah, it's been in the works since FUDCon Toronto.  Complicated stuff.

>  I won't of course study all the code in a large
>  detail, but I'll provide some feedback for the test object (and other 
>  AutoQA-specific files) soon. 
These files should hopefully be very small.

>  We can do such changes, but I can't guarantee it will be included
> in
>  the next release. But, your test works independently of those changes,
>  right? If you prefer, we can include your test in the next release
>  and change the path slightly later. I don't really see a problem in that. 
My test should work independent of its source tree location, yes.  I
just thought it might make for cleaner history to move before merging.
But I do not have a strong opinion.

>  I think you just did it :) If we can cleanly include it in our
> current
>  code (it seems we can), there's no reason why it should not be part of
>  the next release (0.4.4). Unfortunately I don't have a spare bare metal
>  machine to test your code. (I should inquire how to use RHTS). How long
>  does the test run take? Are there any issues we should be aware of? 
Since I did not alter anything outside of tests/anaconda_storage/, I
don't see any problems or issues preventing it from being included in
the next release.

I'll do a test run later and see how long it takes me.  It certainly
does seem like a while since a livecd has to be created first.

- Chris