Automatic Smoketests for the Cloud Images: What to Test?

Sandro "red" Mathys red at fedoraproject.org
Wed Mar 5 11:58:32 UTC 2014


Heads-up: I've taken ownership for the external need "Automatic
Smoketests on Image Build" [0]

Testing an image takes time and resources, and with several images, it
takes several times that. Since that simply doesn't scale - well,
doesn't even work well without scaling - we want to automate the
smoketests using Taskotron. Unfortunately, Taskotron is still in a
very early development phase and having mentioned only just our (most)
basic needs, Tim estimates it won't be good enough for us in the
target timeframe (i.e. by F21 Alpha). There's a slight chance it will
be ready, but even then we still need time to implement the
smoketests.

So if we want to have Taskotron ready for us in time, we need to find
some developer(s) committing some of their resources. Any takers? If
you're interested, head to [1] to learn more, there's also a
contribution guide linked from there. I'm sure tflink in #fedora-qa is
happy to talk to people still interested after reading through the
wiki if there's further questions.

Also, we need to compile our actual needs. What kind of tests do we
want to run? Most basic tests (things that we manually tested in the
past) are probably:
- Was the IP correctly set up?
- Is SSH running and reachable?
- Does the fedora user exist?
- Is the ssh pubkey installed for the fedora user?
- Is the fedora user able to use sudo?
- Is the root account locked?
- Is package installation working?
- Is the firewall disabled?
- Checking cloud-init and systemctl for bootup errors - does that make
sense or is there too much noise?

What else can / should we test? Obviously the tailored images should
be tested for their specific stuff but lets first focus on the common
ground.

Thanks,
Sandro

[0] https://fedoraproject.org/wiki/Cloud_Changelist#External_Need:_Automatic_Smoketests_on_Image_Build
[1] https://fedoraproject.org/wiki/Taskotron


More information about the cloud mailing list