Re: AutoQA PUAP check-in
by James Laska
Howdy folks,
My action items from our last meeting were pretty basic compared to post-bodhi-update, virt-support and resultsdb:
* [jlaska] find a machine where resultdb may run (it can be VM, publicly
accessible preferred)
* [jlaska] set up testing instance of autotest-server
I believe we should be good to go on both the tasks above. I took the second task as an opportunity to tackle ticket#188 and update our autotest to the latest upstream 0.12.0. The autoqa (master+wwoods branch) and autotest-0.12.0-1 that Kamil is using are the lastest builds. I can do another autoqa build now that the master branch has been updated.
Thanks to the infrastructure team, http://autoqa.fedoraproject.org is coming along nicely. That system is an EL5 xen guest, and smooge created the necessary mysql databases for us on Friday. I'd like to deploy the latest autoqa+autotest packages on that system and move our current "production" instance to autoqa.fedoraproject.org within the next 2 weeks.
Thanks,
James
----- "Josef Skladanka" <jskladan(a)redhat.com> wrote:
> Hello gang!
>
> i started to work on a library, which wraps the usage of xmlrpc
> resultsdb interface, so we can easily use it from inside the tests. I
>
> also rewrote the initscripts test so it stores data in resultsdb via
> this interface. I'll edit more of the tests next week. You can find
> these changes in jskladan branch.
>
> Next, I'd like to push this chaged version to the testing machine
> kparal
> uses for experimenting with autotest, so it can actually gather some
> data into the resultsdb, and we can start creating some frontend.
>
> Cheers
>
> joza
>
> _______________________________________________
> autoqa-devel mailing list
> autoqa-devel(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/autoqa-devel
13 years, 10 months
Re: [AutoQA] #87: new post-bodhi-update hook
by fedora-badges
#87: new post-bodhi-update hook
----------------------+-----------------------------------------------------
Reporter: wwoods | Owner: wwoods
Type: task | Status: closed
Priority: major | Milestone: autoqa depcheck
Component: harness | Version: 1.0
Resolution: fixed | Keywords:
----------------------+-----------------------------------------------------
Changes (by wwoods):
* status: assigned => closed
* resolution: => fixed
Comment:
hook is complete and merged into master.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/87#comment:12>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 10 months
[AutoQA] #109: post-bodhi-update watcher
by fedora-badges
#109: post-bodhi-update watcher
----------------------+-----------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: autoqa depcheck
Component: watchers | Version: 1.0
Keywords: |
----------------------+-----------------------------------------------------
the post-bodhi-update hook (see ticket #87) will require a post-bodhi-
update watcher. This should use the bodhi API to get info about newly-
created/changed updates and fire tests.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/109>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 10 months
labels branch - use autotest labels in autoqa
by Kamil Paral
Hello,
I have pushed a new branch "labels". Please do
$ git diff origin/master origin/labels
to view the changes.
This code should enable us to leverage additional autotest
labels (not just the platform ones) for scheduling jobs.
So what we can do with it?
1. We can force a specific test case to be run always
in a virtual machine.
2. We can ensure that a specific test case is always run
only such a machine that has KVM support.
3. We can finally specify which Fedora release must be
installed for our test case to run. Let's take the
"initscripts" as an example - when we want to test *.fc12
package, we need to test it on Fedora 12 machine, and
similarly *.fc13 package on Fedora 13 machine. So we
must define it dynamically according to current input
arguments. We can do that now. (Btw, how the hell did we
do it up till now? I think we've forgotten about it :P)
Everything is defined in test_case/control file, like this:
aq_labels = ['virt', 'fc12']
or like this
aq_labels = ['virt']
aq_labels.append(envr.split('.')[-1]) # add distribution label (like 'fc13')
You can look at rpmlint, rpmguard and initscripts control files,
for testing purposes I have forced different labels to be
required for them.
I have tested the code a little on our staging server, and
I'll do more thorough testing soon (once I tackle all the
VM problems I encountered).
Comments, concerns, improvements welcome.
Thanks,
Kamil
13 years, 10 months
Re: [AutoQA] #88: bodhi_utils module for autoqa lib
by fedora-badges
#88: bodhi_utils module for autoqa lib
---------------------+------------------------------------------------------
Reporter: wwoods | Owner: wwoods
Type: task | Status: assigned
Priority: major | Milestone: autoqa depcheck
Component: tests | Version: 1.0
Resolution: | Keywords:
---------------------+------------------------------------------------------
Changes (by wwoods):
* owner: => wwoods
* status: new => assigned
Comment:
There are a few methods in watch-bodhi-requests.py that should probably be
moved to a common library:
{{{
bodhitime()
parse_bodhitime()
bodhi_list()
}}}
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/88#comment:2>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 10 months
post-bodhi-update early test results
by James Laska
Greetings,
Since I have an extra virt autotest-server setup, I decided to run
through a few post-bodhi-update tests using the autoqa-0.3.5-pre1
package [1].
A few observations, some are probably known/expected, but I'll list them
here for others.
1. I needed to create /usr/share/autoqa/post-bodhi-update/testlist
for /usr/bin/autoqa to recognize the watcher. I just added
rpmlint to test (see step#4 for results)
2. See attached output from running watch-bodhi-requests.py for the
first time. Nothing horrible here. Perhaps reducing the error
output produced by calling load_update_ids() when no cachefiles
are present?
3. Attempting to schedule an autotest job using
watch-bodhi-requests.py fails due to incorrect command-line
arguments (see below). The attached patch resolves that
problem.
4. Re-running the '/usr/bin/autoqa' using the hook patch from
step#3 schedules an autotest job using the attached control
file. Obviously, the job.run_test() call for rpmlint, doesn't
use the same arguments that post-bodhi-update is creating. So
'rpmlint' might not be a good test. I'll try using the
'depcheck' test next.
Thanks,
James
[1]
https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000687.html
13 years, 10 months
Re: post-bodhi-update early test results
by Kamil Paral
----- "Will Woods" <wwoods(a)redhat.com> wrote:
> On Tue, 2010-06-15 at 15:59 -0400, James Laska wrote:
> > Greetings,
> >
> > Since I have an extra virt autotest-server setup, I decided to run
> > through a few post-bodhi-update tests using the autoqa-0.3.5-pre1
> > package [1].
> >
> > A few observations, some are probably known/expected, but I'll list
> them
> > here for others.
> >
> > 1. I needed to create
> /usr/share/autoqa/post-bodhi-update/testlist
> > for /usr/bin/autoqa to recognize the watcher. I just added
> > rpmlint to test (see step#4 for results)
>
> I was thinking - maybe we should have a really stupid
> "helloworld"-style
> test that just basically logs its inputs and reports success. This
> would
> allow us to test new hooks without needing to find an appropriate
> test.
>
> > 2. See attached output from running watch-bodhi-requests.py for
> the
> > first time. Nothing horrible here. Perhaps reducing the
> error
> > output produced by calling load_update_ids() when no
> cachefiles
> > are present?
>
> Yeah, I intended that we would remove/quiet that code once we were
> satisfied that the watcher wasn't doing anything stupid.
>
> > 3. Attempting to schedule an autotest job using
> > watch-bodhi-requests.py fails due to incorrect command-line
> > arguments (see below). The attached patch resolves that
> > problem.
>
> Ah doh! I forgot to change that. Patch looks fine; please apply.
>
> > 4. Re-running the '/usr/bin/autoqa' using the hook patch from
> > step#3 schedules an autotest job using the attached control
> > file. Obviously, the job.run_test() call for rpmlint,
> doesn't
> > use the same arguments that post-bodhi-update is creating.
> So
> > 'rpmlint' might not be a good test. I'll try using the
> > 'depcheck' test next.
>
> So maybe let's write that "helloworld" test, and try that out instead
> of
> trying to get depcheck working first?
Excellent idea.
>
> All it would need to do is something vaguely like:
>
> def run_once(self, *args, **kwargs):
> msg = "args = %s\nkwargs = %s" % (args, kwargs)
> self.results = msg
> print msg
> # maybe email the message here just to be sure email is working
> # etc.
>
> ...except writing the control file might be a bit tricky since we
> don't
> necessarily know the key names. hrm.
Let's just pass on whole locals(), shall we?
>
> So Maybe autoqa ought to be writing all the hook arguments into a
> dict
> instead of individual args, so we can pass along all the autoqa
> arguments without knowing their names.
>
> This is a valuable exercise, though - we're starting to have tests
> that
> could apply to multiple hooks (rpmlint for example) and we're going
> to
> need ways to handle that.
Yea, that will be needed for the future. But our current helloworld
can be just dead simple.
>
> For example we might also need to standardize on argument names - so
> (e.g.) --target-tag will always mean "this is the tag that this
> package
> is trying to enter" regardless of whether we're doing post-koji-build
> or
> post-bodhi-update or whatever.
>
> In the meantime, though, we can just write this 'echo_inputs' test
> specifically for post-bodhi-update, and worry about how to make it
> work
> for multiple hooks later.
>
> So. Anyone want to take a crack at writing this test?
I should stop playing with autotest labels in one or two days
so I can write it if it is still not done by then.
Ticket here:
https://fedorahosted.org/autoqa/ticket/195
>
> -w
>
> _______________________________________________
> autoqa-devel mailing list
> autoqa-devel(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/autoqa-devel
13 years, 10 months
