On Mon, 2017-03-06 at 14:06 -0500, Kamil Paral wrote:
> On Mon, 2017-03-06 at 12:00 -0500, Kamil Paral wrote:
> > > If we implement a 'scenario' key, we can then just teach Bodhi to
look
> > > for that and combine it with the item, then maybe if it can't find
one,
> > > treat 'testname-arch' as the scenario (as a fallback for old
results,
> >
> > We don't have that many tests yet, and all of them are basically
> > under our control, so if we end up doing this, I think we should
> > simply push a patch to all of them (should be fairly easy) and avoid
> > any legacy handling.
>
> Well, the case I was thinking of is specifically Bodhi: even if we
> adjust things so that all results *going forward* have a scenario key,
> *existing* results will not, and often updates sit around in Bodhi for
> a long time before going stable. If we make Bodhi only work if the
> result has a 'scenario' key, it'll either break or not be able to
> display test results for older updates, I think. (And I think Bodhi
> shows results even for stable updates, so really, don't we need it to
> be able to handle the existing results for all non-EOL releases?)
I was under the impression that Bodhi would consider the scenario key
only if present, otherwise consider all testcase+item-based results
unique. So it should work both for future and past results (with the
slight exception of disregarding arch-specific results). Am I missing
something? Or are you hinting at the arch-specific results
(depcheck)?
Sorry, yeah, that's exactly the case in point (and older openQA results
that don't include a 'scenario' key, I was planning to teach Bodhi how
to 'synthesize' it for that case). If we remove its existing ability to
'synthesize' the scenario for depcheck results, it'll display older
depcheck results with no scenario key improperly, right?
I was planning to clearly mark the 'synthesis' code as something that's
needed only so long as we care about older results without the scenario
key, so it can be removed in future.
Of course, an alternative would be to go into ResultsDB and amend all
existing older depcheck and openQA results to include a 'scenario' key.
I don't know if that's frowned-upon, though? I don't think it would be
very technically difficult to *do*, all the necessary information
should still be available (though for openQA results I think we may
have to query some of the values from openQA, I don't believe we
forward all the relevant 'scenario' key values to RDB at this
time...probably we should).
> I agree we should implement the 'scenario' reporting for
all systems
> ASAP (should I send a patch for taskotron, if I can figure it out?),
> but I don't think that's actually sufficient, for the above reason.
I can definitely take care of adjusting the core taskotron once we
decide to go full steam ahead. From tasks, it seems only
depcheck/rpmdeplint will be affected (nothing else uses any extradata
fields). Of course ideally we should also document this in
libtaskotron docs and/or resultsdb_conventions. But I'd leave that
until we test it out and realize whether it's working correctly or
not.
Agreed. Thanks!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net