----- "James Laska" <jlaska(a)redhat.com> wrote:
Merging my response for both mails into one. I hope that's not
too
confusing.
Great set of comments! Replying below.
> 3. BODHI_POSTING_COMMENT_SPAN (comment duplication protection)
is
> currently set to 3 days
Does it make sense to store/access this value in autoqa.conf in a
[bodhi] section?
Yes, it may be well a configurable option. That will allow us to easily
change it any time we need. Great idea.
Also, now that I think of it (see below), shouldn't
the bodhi server URL be a configuration option. Unrelated to this
patchset, but this also applies to the koji server URL?
Interesting. That would certainly help those people I spoke with at FUDCon
Zurich. They were interested in AutoQA and they were running their own
Koji instance.
I'll add this to "add support for staging server" ticket, it's somewhat
related.
> +def get_cfg(cfgfile, section, default_conf = {}):
> + '''Get data from config file
> +
> + Args:
> + cfgfile -- config file name
> + section -- section of the config to be retrieved
> + default_conf -- default configuration values
> +
> + Returns:
> + Dictionary containing retrieved data on success.
> + '''
Can we make get_cfg() accept a list of cfg files to try, not just a
single file (see similar example in lib/python/repoinfo.py)? I've
been
meaning to do this for the 'autoqa' script next time I'm in there.
Mainly because it makes running tests/watchers directly from a git
check-out difficult if the code only looks in
'/etc/autoqa/autoqa.conf'.
With repoinfo, it looks for both
['repoinfo.conf,'/etc/autoqa/repoinfo.conf'].
Everything fits together. Yes, that's certainly a good idea. Thanks to my
latest patch we have now config files transferred together with the test.
We can access repoinfo (and other conf files) in the test directory, we
don't need to maintain /etc/autoqa files anymore.
I'll create a patch for repoinfo handling once this patch is in master, but
Martin may prepare the libraries even in this patch. Good idea.
Speaking of, should
the
'autoqa' script to use the new get_cfg() method to access it's conf?
Forwarding to Martin.
> + if not _is_bodhi_testresult_needed(old_result,
comment_time, result):
> + print 'The test result already posted to bodhi.'
> + return True
Does that 'print' statement show up in our autoqa logs so we know
when
it decided not to post feedback into bodhi?
Yes, as with all other test output, this is accessible in the autotest client
logs (stored at autotest server).
> def run_once(self, **kwargs):
> - pass
> + os.chdir(self.bindir) # easiest way for tests to find their
test scripts, config files, etc
>
It looks like there are some changes to lib/python/test.py
(os.chdir(self.bindir)) and the test templates. Are those changes
related to the bodhi comment support? Does changing the PWD for all
our
run_once() tests alter there outcome?
We could have probably mentioned this, it's a notable change. We change now
CWD to self.bindir before any test starts. The reason is fas.conf. It's
located in the test's directory (self.bindir), but autotest sets CWD to
self.outputdir by default. In order for our libraries to be able to read
fas.conf (and other conf files), we need either
1. give them full path
2. pass on self object, so they can extract self.bindir
3. change CWD to self.bindir
We decided to do option 3), because it seemed best. Martin did some testing
and changing CWD did not introduce any problems. Moreover, several tests
did this already internally (inside run_once()). The only drawback is that
we now require to call super.run_once() at the beginning of the run_once()
method (same as for initialize() and setup()).
(Could that be simplified by using a decorator function, Josef?)
> @ExceptionCatcher()
> def run_once(self, baseurl, parents, name, **kwargs):
> + super(conflicts, self).run_once()
> if name:
Same as above, is this related/required by the bodhi comment support?
Also, if this is needed, we'll want to update the documentation?
We adjusted doc/test_class.py.template and we will probably also adjust
wiki documentation, yes. This ticket will be re-assigned to 'docs'
component once the patch has been committed.
> 5. We will need to test this feature more before final release.
Martin
> did a few manual tests, but that's not enough. My best idea is
that
> we could use our development server, deploy the code there,
deactivate
> actual bodhi sending code and just intercept the calls and log
them.
> We would have it running for a week and only then we can be
quite
> sure it will really work once we make a new release and use it
for
> production machine.
lmacken informs me that we can use a staging bodhi instance for
testing
comments (
https://admin.stg.fedoraproject.org/updates). Email
notifications are disabled, so we shouldn't have angry maintainers
hunting us down as we test this new support :)
Oh, that sounds great. The only problem is that the staging server
doesn't seem to share content with the production server, the package
information are quite outdated. That means:
1. We won't probably get any new package update notifications.
2. If we listen to production server and try to send comments to the
staging server, we probably won't find the matching updates.
So, we can't probably hook it up permanently to our AutoQA staging
instance (in the future). But we can certainly use it for
semi-automatically sending a few dozen comments, for different use cases
etc. Great.