AutoQATest work (jskladan branch) update
by Will Woods
So! I've been testing Josef's work on the AutoQATest class and I think
I've got the rough edges filed down so it's basically working as
expected!
You can check out my updates to his work on the jskladan branch - I'm
going to keep testing things on our local setup here until I'm more sure
it's ready, and then I should hopefully be merging that branch on Monday
or Tuesday.
-w
13 years, 8 months
[AutoQA] #118: New test proposal: Python debugability
by fedora-badges
#118: New test proposal: Python debugability
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: minor | Milestone:
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
From Adam Williamson:
I've cut some of the context, but
basically David wants to write a test case for checking whether Python
debugging is possible as intended, and I asked exactly how he wanted it
to be used.
{{{
On Tue, 2010-01-26 at 15:30 -0500, David Malcolm wrote:
> > > Can I request a test case to cover debuggability of the Python
> runtime
> > > please (both in Fedora and in RHEL).
> > >
> > > This is in relation to:
> > > https://bugzilla.redhat.com/show_bug.cgi?id=556975
> > > https://bugzilla.redhat.com/show_bug.cgi?id=558977
> > > https://bugzilla.redhat.com/show_bug.cgi?id=557772
> > >
> > > as there seem to be gcc and gdb issues, which are conspiring to
> make
> > > python impossible to debug.
> > >
> > >
> > > The requirement is: within "gdb python", I must be able to select
> a
> > > PyEval_EvalFrameEx frame, and have the following work:
> > > (gdb) print co
> > > (gdb) print f
> > >
> > > rather that have <variable optimized out>
> > >
> > > so that I can do this:
> > > (gdb) print (char*)(((PyStringObject*)co->co_name)->ob_sval)
> > > to get the function name
> > >
> > > (gdb) print (char*)(((PyStringObject*)co->co_filename)->ob_sval)
> > > to get the source filename
> > >
> > > and
> > >
> > > (gdb) print f->f_lineno
> > > to get the approximate source line number.
> > >
> > > If the above isn't working, it becomes extremely hard to
> meaningfully
> > > debug any issues that arise inside Python.
> This is probably scriptable, and a good candidate for AutoQA and foe
> RHTS.
>
}}}
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/118>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
[AutoQA] #184: Add tags for destructive testing
by fedora-badges
#184: Add tags for destructive testing
----------------------------+-----------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: Virtualization
Component: infrastructure | Version: 1.0
Keywords: |
----------------------------+-----------------------------------------------
We need to add tags into autotest and autoqa tests that will determine
whether test is (may be) destructive. This tag must be propagated and
appropriate test client must be selected. After finishing this test the
test machine must be restored to a previous safe state.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/184>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
[AutoQA] #181: Quick and dirty support for Virtualization
by fedora-badges
#181: Quick and dirty support for Virtualization
----------------------------+-----------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: Virtualization
Component: infrastructure | Version: 1.0
Keywords: |
----------------------------+-----------------------------------------------
The task is to implement quick and dirty support for virtualization for
AutoQA tests. We can already install autotest-client into virt machine,
but there are some tests which must not run inside virt environment (tests
using virt themselves for example). Therefore we must ensure all our tests
are marked properly ("may use virt", "may not use virt"), all our tests
clients are marked properly ("is virt", "is bare metal"), and this
information are propagated from AutoQA to autotest-server and used
properly when selecting appropriate host and scheduling jobs.
This could be quite easy, there is some support for tags in autotest
already.
As an enhancement, we may think about preferences. Do we want to mark some
test that it is preferred to run it in virt, but may also be run on bare
metal? How do we achieve correct behavior for autotest-server for such
task? For example: We may want to prefer executing sanity tests in virt
env, but it's not really mandatory, just desired. Food for thoughts.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/181>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
[AutoQA] #207: post-koji-build watcher does not watch for new updates-candidate builds
by fedora-badges
#207: post-koji-build watcher does not watch for new updates-candidate builds
----------------------+-----------------------------------------------------
Reporter: kparal | Owner:
Type: defect | Status: new
Priority: major | Milestone: Package Update Acceptance Test Plan
Component: watchers | Version: 1.0
Keywords: |
----------------------+-----------------------------------------------------
I have found out that our post-koji-build watches only these tags:
dist-f10 dist-f10-updates dist-f10-updates-testing dist-f11
dist-f11-updates dist-f11-updates-testing dist-f12 dist-f12-updates
dist-f12-updates-testing dist-f13 dist-f13-updates dist-f13-updates-
testing dist-f14
Which is incorrect, we should be watching dist-f*-updates-candidate (and
maybe dist-f14, I don't know). We don't need to watch for the rest of it.
The change happened in commit 580734d4f2aa768aefa1c0498904df2296747918.
See the diff and inspect current contents of repoinfo.conf to grasp the
change.
The result is that we don't currently test almost any packages except f14
ones. Do we have it already deployed on our production machine?
What we can do about it? Well, I hope someone with a deep understanding of
repoinfo stuff knows what's the best approach :) It seems to me that we
could define the *-updates-testing repositories in repoinfo.conf (and make
sure we didn't break something else) and retrieve and select just those in
watch-koji-builds.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/207>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
[AutoQA] #200: Multiple control files / multiple test wrappers
by fedora-badges
#200: Multiple control files / multiple test wrappers
--------------------+-------------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Multi-hook tests
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
For a test to run under multiple hooks, it may need multiple control
files, since we may want to do slightly different things to prepare for
each test. We could have multiple test wrappers and conditionalize the
job.run_test() argument based on the calling hook, for example:
if hook == 'post-koji-build':
job.run_test('mytest_koji', name=name, envrs=envrs, ...)
elif hook == 'post-bodhi-update':
job.run_test('mytest_bodhi', name=name, envrs=envrs, ...)
Alternately, we could have multiple control.HOOKNAME files - one for each
hook the test can be run under. This would also allow us to eliminate the
use of the 'testlist' file to determine the tests that are eligible for
use in each hook - each test would be able to declare how to run itself
under each hook.
This ticket requires deciding on a method for handling multi-hook tests,
documenting that method, and (perhaps) converting existing tests.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/200>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
Update repoinfo.conf for F-14 branch
by James Laska
Greetings,
Per ticket#211, I have updated repoinfo.conf for the new F-14 branch. Instructions for updating repoinfo.conf are on the wiki at https://fedoraproject.org/wiki/How_to_update_AutoQA_repoinfo.conf. I have tested this change and confirmed that watch-koji-builds.py goes from monitoring these repos:
Looking for builds with tags dist-f10-updates-candidate
dist-f11-updates-candidate dist-f12-updates-candidate
dist-f13-updates-candidate dist-f14
To monitoring for koji builds in these repos:
Looking for builds with tags dist-f10-updates-candidate
dist-f11-updates-candidate dist-f12-updates-candidate
dist-f13-updates-candidate dist-f14-updates-candidate dist-f15
Thanks,
James
13 years, 8 months
