Taskotron and Cloud Image tests

[Vitaly Kuznetsov]

Hi Tim & all,

in Cloud WG we have 'Automatic Smoketests on Image Build' task:
https://fedorahosted.org/cloud/ticket/38
AFAIK you have somehting similar among future Taskotron goals. It would
be nice if we can collaborate here. By writing this message I just want
to start the conversation :-)

For RHEL cloud image checks we have special tool called 'valid':
https://github.com/RedHatQE/valid , it can be reused for Fedora as
well. The questions are: how do you see the integration and how can
someone (me?) work on this (non-existent atm) part of Taskotron.

I can see two possible approaches:
1) 'Easy'. We create special type of task (sorry if I'm not that precise
with terminology) which is being executed on static slave node which
runs valid (in black-box mode) on newly uploaded image, grabs the result
and returns it back.

2) 'Hard'. We work on 'ephemeral task clients' (term from
http://tirfa.com/an-initial-idea-for-taskbot.html). The benefit here is
that in that case we can run any test on a cloud VM. This approach will
require some optimization from the very beginning, at least:
1. One VM should be user for multiple tests (so several dozens of image
tests won't require several dozens of VMs created)
2. Tests should be able to 'control' VM (e.g. use cloud-specific
features: attach device, reboot VM,...) - that can be workarounded by
providing cloud credential inside the VM itself but that doesn't sound
that attractive.

Yesterday I've tried setting up Taskotron (according to
http://roshi.fedorapeople.org/dexy-themed/) on 3 F20 VMs in EC2 and
actually I've failed (some ansible playbooks are out-of-sync with some
git repos, e.g. 'master playbook' is failing with 'stderr: cp: cannot
stat '/home/qaadmin/trigger/jobtrigger.py': No such file or directory'
in 'TASK: [copy fedmsg config]'. I totally understand these instructions
and repos are rather proof-of-concept and can be slightly outdated but
I want to ask about 'minimalistic one-host dev environment' setup
anyway. Can 'Host' be eliminated from the setup completely and can we
merge Master and Slave (in case we'll need it for 'cloud') on one host?
That would help a lot.

I'm looking forward to hearing from you.

-- 
  Vitaly Kuznetsov