Rawhide update gating on openQA tests will be enabled Wednesday
Hey folks! So, as per https://pagure.io/fesco/issue/3011 , we plan to
enable gating of Rawhide updates on openQA test results on Wednesday.
What does this mean?
For many updates, nothing at all: only updates on the critical path
will be gated.
For most critical path updates: instead of the build(s) being tagged
and going into the buildroot immediately after the build, the update
will go into a "waiting" state for about two hours, then - when the
tests have all passed - the builds will be tagged and put in the
buildroot. This is just the same as for branched and stable updates.
For the few critical path updates where a gating test fails: the
packages will not be tagged or reach the buildroot. The QA team should
notice the failure, investigate, and either fix the problem or file
some kind of issue report linked from the update on Bodhi within 24
hours. If this does not happen, or the update needs urgent attention,
please contact the QA team on Fedora Chat, or by email to test@ .
If you believe a gating failure may be a false alarm, you can click the
"Re-Run Tests" button in Bodhi to have the test re-run. If you are
*very very sure* it is a false alarm, and the need is urgent, you can
click the "Waive Results" button in Bodhi to waive the failure and have
the update tagged, but please only do this if there's really no
alternative. Please note that waiving a failure comes with a risk that
the same failure will happen for *every subsequent update*, causing
each of them to be gated, because the waived update will immediately
reach the buildroot, and openQA uses the buildroot as its "base"
package set for testing. Really, please don't do this unless you're
very sure!
If you have a big set or chain of builds to do and you don't want to
have to wait two hours between dependent builds: use a side tag! Side
tags are self-service now. You can just request one, it will be created
within minutes, and you can do your builds on the side tag. No gating
applies to side tags. When you're done, create an update from the side
tag in Bodhi (the web UI has support for this), and it will be tested
and gated as a single unit (the openQA tests will run once on the
update as a whole). See
https://docs.fedoraproject.org/en-US/rawhide-gating/multi-builds/ for a
walk-through of this process.
The display of gating status and test results on the Bodhi web
interface should be clear, consistent and accurate. If you find it is
not in any way, please file a ticket on Bodhi and tag me on it.
If this turns out to be a bad idea, we can and will easily revert it,
so please don't panic. :)
Thanks everyone!
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net