8:09 a.m.
Hello!
There are some cool new features in Fedora CI that we would like to tell
you about.
First of all, Fedora CI now supports running tests using the new tmt
format. The tmt tool provides better user experience for enabling, creating
and running tests across different environments (e.g. vm, container,
localhost). The same configuration can be used for enabling tests in
Packit, Fedora CI and RHEL CI. See [1] to get a quick start.
This functionality is available for Rawhide and for dist-git pull requests.
The older STI format is, of course, still supported and it takes precedence
over tmt. If you’d want to switch over to tmt, you will need to rename or
delete the tests*.yml files used by STI.
Next there are 3 generic tests that automatically run on all Rawhide
builds. Note not all of them are completely new — you’ve likely seen some
of these tests before. What’s really new is that all these generic tests
run in Testing Farm [2] now. Testing Farm is a new test execution service
that provides an infrastructure that should be much more stable and as a
result, less flakiness overall.
These generic tests are:
-
rpmdeplint
-
rpminspect
-
installability
If you’d want to know more about any of those tests, please check out the
Fedora CI wiki page [3].
And last but not least, here are some real world examples of (potential)
problems that the tests discovered in the recent Rawhide builds:
[rpmdeplint] undeclared file conflicts:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpmdeplint-...
[rpminspect] relaxed permissions:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-...
[installability] package cannot be installed:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/installabil...
All this was a multi-team effort and I'd like to thank everyone who
participated :)
Thanks,
Michal
[1]: https://docs.fedoraproject.org/en-US/ci/tmt/
[2]: https://testing-farm.gitlab.io/api/
[3]: https://docs.fedoraproject.org/en-US/ci/generic_tests/