Excerpts from Adam Williamson's message of 2017-05-02 16:15 -07:00:
So, first the good news: fundamentally, the concept works. I can
tell
Bodhi to query 'all bodhi_update results for the item FEDORA-2017-
4e95959f0d' and then display (only) the latest result for each
'scenario' that appears in the whole set of results.
I think this might be the fundamental problem here. I think the
ResultsDB "item" should not be an update, but the set of packages in an
update. If someone edits the update to change the set of packages in it,
the previous results are invalidated and new tests are kicked off.
Putting aside the issue of how to actually make the "item" be a set of
packages -- I think this would eliminate all of these issues with OpenQA
scenarios.
OpenQA would simply report results with test cases like this, with all
your scenarios and flavours and so forth as different permutations in
the test case name:
graphical_installation.fedora.26.server.x86_64.blahblah
graphical_installation.fedora.26.workstation...
and the existing logic for latest-result-by-testcase will work fine with
no special cases needed in Bodhi. Bodhi would just query for the
ResultsDB "item" consisting of all packages currently in the update.
This also eliminates the problem where an update is edited, and Bodhi
queries for item FEDORA-2017-4e95959f0d but it sees older test results
based on the packages that were previously in the update because the new
test runs are not complete yet.
If you go down that route then it means the *real* problem to solve is
how to represent an "item" which is a set of packages rather than
a single package.
--
Dan Callaghan <dcallagh(a)redhat.com>
Senior Software Engineer, Products & Technologies Operations
Red Hat