Taskotron and Cloud Image tests
by Vitaly Kuznetsov
Hi Tim & all,
in Cloud WG we have 'Automatic Smoketests on Image Build' task:
https://fedorahosted.org/cloud/ticket/38
AFAIK you have somehting similar among future Taskotron goals. It would
be nice if we can collaborate here. By writing this message I just want
to start the conversation :-)
For RHEL cloud image checks we have special tool called 'valid':
https://github.com/RedHatQE/valid , it can be reused for Fedora as
well. The questions are: how do you see the integration and how can
someone (me?) work on this (non-existent atm) part of Taskotron.
I can see two possible approaches:
1) 'Easy'. We create special type of task (sorry if I'm not that precise
with terminology) which is being executed on static slave node which
runs valid (in black-box mode) on newly uploaded image, grabs the result
and returns it back.
2) 'Hard'. We work on 'ephemeral task clients' (term from
http://tirfa.com/an-initial-idea-for-taskbot.html). The benefit here is
that in that case we can run any test on a cloud VM. This approach will
require some optimization from the very beginning, at least:
1. One VM should be user for multiple tests (so several dozens of image
tests won't require several dozens of VMs created)
2. Tests should be able to 'control' VM (e.g. use cloud-specific
features: attach device, reboot VM,...) - that can be workarounded by
providing cloud credential inside the VM itself but that doesn't sound
that attractive.
Yesterday I've tried setting up Taskotron (according to
http://roshi.fedorapeople.org/dexy-themed/) on 3 F20 VMs in EC2 and
actually I've failed (some ansible playbooks are out-of-sync with some
git repos, e.g. 'master playbook' is failing with 'stderr: cp: cannot
stat '/home/qaadmin/trigger/jobtrigger.py': No such file or directory'
in 'TASK: [copy fedmsg config]'. I totally understand these instructions
and repos are rather proof-of-concept and can be slightly outdated but
I want to ask about 'minimalistic one-host dev environment' setup
anyway. Can 'Host' be eliminated from the setup completely and can we
merge Master and Slave (in case we'll need it for 'cloud') on one host?
That would help a lot.
I'm looking forward to hearing from you.
--
Vitaly Kuznetsov
10 years
qa host firmware updates
by Tim Flink
I mentioned this a while ago but just recently figured out the
process details and got the permissions I needed to complete the work.
Our hosts are getting pretty old and most of them haven't seen a
firmware update since they were installed 3-4 years ago. This hasn't
been a problem for most of the hosts but one of them (qa06) has always
been a special case due to how its' disk shows up.
The firmware was updated on qa06 in January and since we couldn't get
the same version that's on the rest of the qa hosts, we ended up using
the newest available versions.
I'd like to get all our hosts at the same firmware versions but the
process of doing that update is a bit painful and can't be done while
the host is in use (requires re-imaging and several reboots -
thankfully all the hosts are ansible controlled and the recovery is
mostly a matter of waiting for the playbooks to finish) so I'm not
planning to do them all at the same time.
While we still have some spare capacity (ie, before we deploy
taskotron), I'm planning to update the firmware on each of the qa
machines by taking them down one at a time, doing the updates and
reconfiguring them for autoqa, autoqa-stg or beaker.
The only place where this will cause a disruption is in beaker, since
we only have one host allocated to that. Both autoqa and autoqa-stg
have multiple hosts allocated so while they might slow down a bit,
they'll both continue to function during the updates.
Since nobody is using beaker at the moment, I'm going to tackle that
host next. It should be back up by the end of the day. I'll update the
thread when I've finished all the upgrades.
Tim
10 years
Project Locations and Basic Setup
by Tim Flink
I think most of this came up in an earlier thread but I'm not finding it
right now, so I'm starting a new one.
I'm proposing that we make all the taskotron-related projects as
similar as we can with regard to git repo setup, location and how we do
reviews. If we keep them all slightly different, it makes asking for
contributions more difficult and the contributions guide (which still
needs to be written :-/) more complicated.
To be specific, the repos that I'm including in this are:
- libtaskotron
- taskotron-trigger
- resultsdb
- resultsdb_frontend
- resultsdb_api
- pytap13
- fake_fedorainfra
The settings that I'm proposing for all those repos are:
- use gitflow and set 'develop' as the default branch
- host projects under bitbucket/fedoraqa
* makes it easier to say "find our projects at this url" instead of
"find projects X,Y and Z here. find A and B here"
- code submissions and issues/tasks are tracked through phabricator
Any objections to doing this to all the repos I mentioned?
Tim
10 years
QA Infrastructure Downtime Today
by Tim Flink
In case anyone missed the announcement on devel@, there is an infra
outage scheduled for today starting at 21:00 UTC and lasting up to 4
hours.
AutoQA and beaker will be partially unavailable during this time - I'm
going to take the opportunity to do some updates and maybe some host
rebuilds.
I'll send out an email when everything is back up and the outage is
complete.
Tim
10 years
Taskotron Repos
by Tim Flink
As I'm working more on the ansible playbooks to deploy Taskotron, I'd
like to get a repo setup for our builds until we have stuff in Fedora
proper.
I think we have 3 options: copr, fedorapeople and qadevel.
copr would probably be the easiest of the options - that takes care of
building, repo creation and hosting. The only disadvantage that I see
to copr is that it limits the number of builds available [1] to the
latest successful build and anything else < 14 days old.
[1]https://fedorahosted.org/copr/wiki/UserDocs#Howlongdoyoukeepthebuilds
fedorapeople is what we've used in the past for both blockerbugs and
autoqa. We still need to manage repo content and do the building but it
is a system for hosting the repo which we don't have to maintain.
When I setup the qadevel machine for phabricator, I planned to also use
the machine for at least CI and possibly some docs and repo hosting.
While I think this might be a good long-term solution for integration
with CI and possible continuous deployment in dev/stg, I'm not sure this
would be the best choice right now due to the amount of work which
would be required to get everything running.
Yet another option would be a hybrid approach (which is what I've been
doing for phabricator) would be to build the rpms with copr and mirror
to fedorapeople to keep more builds around. The process is a bit of a
pain but it can be less painful than building everything locally.
I'm of the opinion that either a buildscript or copr and fedorapeople
repos is going to be the best option for the moment. Any other thoughts?
Tim
10 years