3:15 a.m.
hey folks! I mentioned this to jskladan on IRC, but just for the
permanent record, I'm working on optional crash report submission for
openQA.
at first I had the workers clicking through the graphical report
submission process, but that has several problems:
a) needles and keypresses and blah
b) workers don't actually know the job ID or URL, so can't include it
in the bug report
c) requires inventing some kind of way to get a BZ username and
password into the workers without it being logged (doable, but just
unnecessary work, when libreport-plugin-bugzilla already has this set
up)
so instead I'm doing it in report_job_results.py in
openqa_fedora_tools. It actually builds off D310, Jan's improvement to
upload the contents of /var/tmp after a crash.
Given a job_id, we check if there's a var_tmp.tar.gz for that job, and
if there is, we look for libreport 'problem directories' inside it. If
we find any, we extract them from the tarball and run 'reporter-
bugzilla -d (directory)' on them.
That's really it in a nutshell, the rest is just error checks and glue
and frills. There's an attempt to include the web UI job URL in the
bug report for new crash reports (though so far I've been testing with
a problem directory that shows up as a dupe of an existing report, so I
haven't tested this yet), and we capture the IDs of the bugs reported.
I also refactored the reporting functions a bit to avoid code
duplication between calling report_job_results directly and using it
from openqa_trigger, and made it possible to specify the openQA URL in
a config file (so you can do result reporting from a system other than
the openQA host itself - like, fr'instance, a Fedora system with
libreport-plugin-bugzilla installed...)
To test it out you need a job in some openQA instance which has a
var_tmp.tar.gz with a crash directory inside it: I've been testing
with https://openqa.happyassassin.net/tests/2736 . You also need to
put a valid BZ username and password in
/etc/libreport/plugins/bugzilla.conf and, unless you're running on the
openQA host itself (there *are* libreport packages for openSUSE in
some OBS repository, but I haven't tried them), you'll want to create
/etc/openqa_fedora.conf with this content:
[site]
url = https://openqa.happyassassin.net
(or whatever URL is appropriate).
Then you can do this:
python report_job_results.py --crashes 2736
(or whatever the job ID is).
This probably still needs a bit more testing and polish before I
submit it as a differential, but I wanted to give people a heads-up
that I was working on it and explain the general design. My current
patch (against 'develop' branch, to which I've merged the 'live' work
now) is attached.
In case you're wondering what happens with duplicate reports: I tested
and it seems like 'not a lot'. When calling reporter-bugzilla in this
way, if the crash has already been reported, it will only generate BZ
activity if the BZ account in question isn't already on the CC list:
it will add it. But if the BZ account is already on the CC list, it
doesn't change the bug at all, it doesn't add the extra comment saying
'another user encountered this issue'. I checked libreport and it
actually only does that when some comment text has been provided, and
we aren't providing one, so it gets skipped.
If we're still worried about noise on dupes it *is* possible to test
if a bug is a dupe by checking the output of:
reporter-bugzilla -h $(cat duphash)
and completely skip the report submission step if it is, and I
actually had that written, but took it out as it seemed unnecessary.
Easy enough to put it back if we want to, though.
In the current version of the patch things are set up so that
openqa_trigger current or openqa_trigger all or openqa_trigger compose
--submit-results runs will try and report all crashes, but it's
absolutely trivial to change that if we only want to report crashes
via a separate invocation.
Comments welcome!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net