#284: depcheck does not produce output for some pending updates
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: defect | Status: reopened
Priority: major | Milestone: Hot issues
Component: tests | Resolution:
Keywords: |
--------------------+-------------------------------------------------------
Comment (by jlaska):
Replying to [comment:10 kparal]:
Exactly. I'd rather send smaller amount of test results, but
correct,
then send larger amount of test results, but possibly incorrect.
Certainly, someone closer to depcheck internals should speak to this, but
from a high-level ... I don't see any problems with reporting PASS for any
successful builds in that test run. As long as the untested builds aren't
also getting PASSED, we aren't introducing closure problems into the
repos.
I'm afraid that is something a person cannot decide (in case of
depcheck
test). If he could, he would have done that already and we wouldn't have
needed depcheck test in the first case.
Yeah, fair point.
If I look at the problem through package maintainers eyes: "My
package
received FAILED/NEEDS_INSPECTION result, noting that I should review the
test output, but due to a bug in the test, there is no output I can
review. So why did I even receive that warning? It's the problem of
AutoQA, not mine. I can't do anything about it, it just annoys me."
Yeah, if it's clearly a test bug ... and there is no way a maintainer
could determine the cause of the failure by inspecting the logs ... I
could see how this would be unactionable (word?) for the maintainer.
I don't think it's a good idea to send maintainers
notifications about
failures in our test. It's better not to send at all, rather than send
something they can't do anything about.
We're still planning to only send notification when all tests for a
package have finished, right? If so ... this wouldn't be an additional
notification.
You make great points Kamil. I don't want to send maintainers nonsense
results where they can't figured out I'm hoping there is a way we can not
introduce more notifications, but also note that the test did not PASS or
FAIL (e.g. NEEDS_INSPECTION). When I look at bodhi updates that are
missing results, it makes me wonder whether there are
scheduler/test/reporting bugs in AutoQA. Assuming publishing a
NEEDS_INSPECTION result doesn't increase email notification, I'd prefer
getting a NEEDS_INSPECTION result, instead of no result at all.
--
Ticket URL: <
https://fedorahosted.org/autoqa/ticket/284#comment:11>
AutoQA <
http://autoqa.fedorahosted.org>
Automated QA project