Hello,
I would like to introduce a CI/CD workflow for Fedora distgits around
Pull Requests my team and I have started to develop.
We chose to use Zuul [1] (the CI engine) to implement this workflow because
we have strong experience with it and some of its features could help a lot
in the packaging context. To list some of them, Zuul is cross-repository
aware, meaning that dependencies between Pull-Requests can be defined
(the job workspace is populated with the dependent changes). Zuul provides a
way to share artifacts between jobs meaning a parent job can do a
scratch build on Koji and child jobs can use the built artifacts (rpms) to
run various validations. Also Zuul comes with a component called Nodepool
that helps handle VMs or containers to manage F30/31/Rawhide nodes for jobs.
A more exhaustive list of features is available here [2].
For a couple of months, we have been experimenting with packaging and
validation jobs for Fedora's distgits. The idea is to build a flexible
packaging workflow for packagers willing to use Pull Requests on Pagure. The
Pull Request system offers a place to wire Zuul jobs with the PR life cycle.
Indeed, jobs are triggered based on the Pagure Pull Request status
(PR opened/changed/merged). At the moment, the most complete workflow we
have implemented is as follows:
- When a PR is opened, Zuul runs a Koji scratch build job, then in parallel,
it runs a linter, rpminspect, and test (embedded functional tests/STI) jobs.
- When a PR is approved, (when the metadata tag "gateit" is set by one of
the distgit admins) and the CI status is green, Zuul merges the PR.
- When the PR is closed and merged, Zuul runs the regular Koji build.
Another great feature of Zuul is how changes to CI configuration can be done.
In fact, anyone can propose changes via Pull Request to a Zuul job, a project
pipeline template, ... and see that change run without affecting the rest of
the CI. This makes the CI more robust and more user friendly as everyone can
be involved in the CI configuration.
To give a bit of insight about what my team and I are doing: we maintain
https://softwarefactory-project.io and
https://review.rdoproject.org that
both use Zuul to provide the CI for more than 1800 repositories
(~95% of distgits). Our instance of Zuul runs (among others) the Pagure
driver [3] for
pagure.io and
src.fedoraproject.org.
For the moment, just three distgits are configured to let Zuul manage the
packaging workflow via PR, but we are looking for early adopters to
experiment and help improve the jobs and workflows. The process to attach
a distgit on Zuul is described here [4]. Having Zuul manage the PR workflow
is not incompatible/conflicting with a regular direct push workflow.
We would be really happy to help anyone willing to be involved :)
Fabien
[1]:
https://zuul-ci.org/[2]:
https://fedoraproject.org/wiki/Zuul-based-ci#What_is_Zuul.2FNodepool[3]:
https://zuul-ci.org/docs/zuul/admin/drivers/pagure.html[4]:
https://fedoraproject.org/wiki/Zuul-based-ci#How_to_Zuul_attach_a_Pagure_repository_on_Zuul