Test gating enabled in Bodhi

[Pierre-Yves Chibon]

Good Morning Fedorans!

On Thursday, a new version of Bodhi was deployed that enabled Bodhi to
gate updates based on test results. You may notice a "Test Gating
Status" message in the right have side of the page.

One thing to know about this is that there is currently a confusing
issue where Bodhi will say that the tests have failed when the tests
haven't finished running[0]. We are working on solving that issue, but
for now you can just wait a while and it should report the result once
all the tests have finished.

There are three tests that must pass in order for updates to go to stable:

0. dist.depcheck - to make sure the update's dependencies are available.
1. dist.abicheck - to make sure the update's ABI remains stable in a
   given Fedora release.
2. AtomicCI pipeline - packages that are part of the Atomic Host *and* include
   in their dist-git tests, must have these tests passed in the AtomicCI pipeline

This last requirement only concern packages that are in the Atomic Host while
the first two is enforced for all packages.

We are also working on another CI pipeline that will allow running tests stored
in dist-git for all packages without discrimination.

You are in control of these three tests. The first two can be fixed by
making changes in the spec file, and the latter test is controlled via
the tests in your repo.

Finally, if it turns out you need to push an update through despite of the test
results, you can do so using waiver-cli (dnf install waiverdb-cli). We are
working on integrating this into Bodhi itself, making this easier.

We have started a wiki page to store all of this information and that we will
keep up to date as improvements are done or for any frequent questions coming up:

  https://fedoraproject.org/wiki/CI/gating_updates

Please accept my apology in the delay in writing this message. It should
have been sent last week when it was enabled, and it got lost.


Thanks for your attention,
Pierre

[0] https://github.com/fedora-infra/bodhi/issues/2124