Automatic Smoketests for the Cloud Images: What to Test?
vitty at redhat.com
Fri Mar 7 09:37:52 UTC 2014
"Sandro \"red\" Mathys" <red at fedoraproject.org> writes:
> Heads-up: I've taken ownership for the external need "Automatic
> Smoketests on Image Build" 
> Testing an image takes time and resources, and with several images, it
> takes several times that. Since that simply doesn't scale - well,
> doesn't even work well without scaling - we want to automate the
> smoketests using Taskotron. Unfortunately, Taskotron is still in a
> very early development phase and having mentioned only just our (most)
> basic needs, Tim estimates it won't be good enough for us in the
> target timeframe (i.e. by F21 Alpha). There's a slight chance it will
> be ready, but even then we still need time to implement the
I just want to point out the fact that for RHEL/Fedora/etc image testing
we have the following tool:
We were using it to test Fedora images since F18.
To be honest I'm not that familiar with Taskotron but maybe we can
> So if we want to have Taskotron ready for us in time, we need to find
> some developer(s) committing some of their resources. Any takers? If
> you're interested, head to  to learn more, there's also a
> contribution guide linked from there. I'm sure tflink in #fedora-qa is
> happy to talk to people still interested after reading through the
> wiki if there's further questions.
> Also, we need to compile our actual needs. What kind of tests do we
> want to run? Most basic tests (things that we manually tested in the
> past) are probably:
> - Was the IP correctly set up?
> - Is SSH running and reachable?
> - Does the fedora user exist?
> - Is the ssh pubkey installed for the fedora user?
> - Is the fedora user able to use sudo?
> - Is the root account locked?
> - Is package installation working?
> - Is the firewall disabled?
> - Checking cloud-init and systemctl for bootup errors - does that make
> sense or is there too much noise?
> What else can / should we test? Obviously the tailored images should
> be tested for their specific stuff but lets first focus on the common
In addition to that current image validation has the following:
- testing different instace types
- empty bash history
- selinux contexts test
- running services
- package set
- EBS test
- Ephemeral test
- available cores/memory test
- userdata handling (cloud-init)
- several bugs test (regressions)
- ... (full test list is here: https://github.com/RedHatQE/valid/tree/master/valid/testing_modules)
>  https://fedoraproject.org/wiki/Cloud_Changelist#External_Need:_Automatic_Smoketests_on_Image_Build
>  https://fedoraproject.org/wiki/Taskotron
> cloud mailing list
> cloud at lists.fedoraproject.org
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
More information about the cloud