On Fri, 2021-05-14 at 14:40 -0500, Michael Catanzaro wrote:
So this seems like a good idea, but I notice that all tests are
marked
as failed until the results arrive. This leads to incorrect failure
emails and incorrect UI indicating lots of test failures where none
exist. Doesn't seem ready for production yet.
I don't know about emails, but the UI isn't indicating a failure, it's
indicating a missing result. This *is* what it shows if you read it
carefully. It doesn't say a test failed.
If emails are getting sent out in this situation we should probably fix
that, but I don't think you can do much about it in Bodhi UI besides
possibly tweak the representation to be a bit clearer about the
difference between a failed test and a missing result. The fact that a
result is missing is significant information and is the reason the test
cannot be gated; it needs to be communicated.
This part isn't any different from existing gating on Fedora CI tests,
AFAIK (although those may run faster and so the state exists for less
time).
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net