Instalatron: Anaconda testing framework
Tim Flink
tflink at redhat.com
Tue Jul 19 16:48:31 UTC 2011
On 07/19/2011 09:49 AM, Sergio Rubio wrote:
> Greetings!
>
> Posted this to the wrong mailing list (David Cantrell suggested using
> this one instead).
>
> Linking to avoid re-typing :), my apologies:
>
> https://www.redhat.com/archives/anaconda-devel-list/2011-July/msg00090.html
>
> In a nutshell, we are looking for better ways to test our Anaconda
> based installer and I thought that speaking to the real guys and
> sharing would be a good way to start.
>
> Any feedback is greatly appreciated.
Cool, we're always looking for ways to do better testing. As James said,
honqing and twu are more actively working on the installer automation
but I have a couple of design questions.
Why virtualbox?
- I was under the impression that VB didn't always play nice with the
fedora kernels and was a reason to use KVM/qemu instead
- Then there's the whole licensing issue ...
How well does it adapt to small changes in visual layout?
- The scripting looks very rigid to me which is fine for things that
don't change very often
- Do the scripts have to be completely re-done for small visual
changes or is it possible to just change a portion?
How often are you seeing false positives/negatives?
- ImageMagick's 'compare' seems awfully sensitive to me, have you seen
many problems with the image comparison logic?
- If I'm reading the code correctly, it seems to be looking for an
exact match
How are you defining pass/fail?
- From my very quick scan of the code, it looks like
PASS -> no crash during installation
FAIL -> image comparison failures, unexpected events, crash
- Is there any verification that the VM boots after install?
How do you handle crashes?
- Are you collecting anaconda stack traces on crash?
- Otherwise, is the VM in a state that allows for data collection if a
crash occurs?
Another thing that you may be interested in is OpenSuSE's openqa project
[1] and specifically, os-autoinst [2].
Thanks,
Tim
[1] http://openqa.opensuse.org/
[2] http://www.os-autoinst.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 553 bytes
Desc: OpenPGP digital signature
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20110719/29bcaa1c/attachment.bin
More information about the test
mailing list