[atomic-wg] Issue #312: Enabling CI/CD: Experiment with building Atomic Host out of a module
dustymabe reported a new issue against the project: `atomic-wg` that you are following: `` This started as an [email](https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2017-Augu...) to the atomic-devel list. Moving into a ticket for further discussion and tracking of experimentation of building Atomic Host out of a module. Summary of bullets from the email:
Here are some of the things we want to be able to do:
- 1 Have a rigorous definition (including specific versions, buildroot) of what goes into an Atomic Host, including dependencies.
- 2 Triggering the CI/CD pipeline based on a change to definition of what goes into Atomic Host.
- 3 A way to revert a package in the Atomic Host compose to an earlier version.
- 4 A place to store higher level tests along with rigorous definition of Atomic Host, including them being versioned.
- 5 Landing multiple changes that need to land together to pass testing.
Out of that list we think we require that:
- 1R The definition of what is composed into Atomic Host artifacts should include specific versions of packages, and all dependencies included.
- 2R The definition of what is composed in an Atomic Host should be stored in a git repository so that changes can be detected easily. The CI/CD pipeline can be triggered off of changes to this reposiroty.
- 3R A mechanism to make a future composed Atomic Host artifact, contain an earlier (in RPM NVR parlance) version of a package.
- 4R The high level functional Atomic Host tests should live in the same git repository with the rigorous definition of what goes into an Atomic Host.
- 5R A mechanism to tell the CI Pipeline that multiple dist-git repository changes (i.e. multiple changing RPMs) should be built and tested together.
Note - @walters also sent [an email](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...) to a broader audience about modularity and Atomic Host that is worth a read. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
dustymabe added a new comment to an issue you are following: `` @psabata, from modularity team, has [volunteered](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...) to help us do a proof of concept on ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
The issue: `Enabling CI/CD: Experiment with building Atomic Host out of a module` of project: `atomic-wg` has been assigned to `psabata` by dustymabe.
walters added a new comment to an issue you are following: `` There's a lot of bugs to work through. I'm trying a build locally, and hitting a lot of variations of
https://github.com/cockpit-project/cockpit/issues/7510
But I'm a bit confused since some of the ones failing locally appear to be succeeding in koji; maybe newer RPM? One thing I haven't found yet is - the base-runtime etc. module is f26 only, is there a version of the host-and-platform somewhere?
Here's one that's currently very mysterious to me: https://koji.fedoraproject.org/koji/buildinfo?buildID=955412 It looks like ostree's use of `AC_USE_SYSTEM_EXTENSIONS` is failing to define `_GNU_SOURCE` somehow.
There's other ones, like `p11-kit` fails with:
RPM build errors: File not found: /builddir/build/BUILDROOT/p11-kit-0.23.7-3.module_55aa6181.x86_64/usr/share/man/man1/trust.1.gz
`selinux-policy` dies with:
libsepol.policydb_read: policydb version 31 does not match my version range 15-30
`libxml2`:
RPM build errors:
File not found: /builddir/build/BUILDROOT/libxml2-2.9.4-4.module_55aa6181.x86_64/usr/lib64/python3*/site-packages/__pycache__/*py*
etc.
``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` Could you elaborate what exactly are you doing? Are you using `mbs-build local` for local tests?
The `base-runtime` module is indeed `f26`-only; it's being replaced by `host` and `platform` in `f27`. The `atomic` module depends on `bootstrap:master` for build, so that's the one `mbs-build local` will download for you -- beware, it's huge. Oh that and `shim:master`.
I can't comment on the build failures; I would need to dive into it some more. Sometimes mysterious things happen :/ ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` I've updated some of the packages and made slight changes to the buildroot -- will kick off another build. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` The current breakage status:
selinux-policy https://koji.fedoraproject.org/koji/taskinfo?taskID=21281956 libtirpc https://koji.fedoraproject.org/koji/taskinfo?taskID=21281972 kernel https://koji.fedoraproject.org/koji/taskinfo?taskID=21282131 cockpit https://koji.fedoraproject.org/koji/taskinfo?taskID=21282552 python-urllib3 https://koji.fedoraproject.org/koji/taskinfo?taskID=21282493 oci-systemd-hook https://koji.fedoraproject.org/koji/taskinfo?taskID=21281537 setools https://koji.fedoraproject.org/koji/taskinfo?taskID=21282943 sqlite https://koji.fedoraproject.org/koji/taskinfo?taskID=21283323 curl https://koji.fedoraproject.org/koji/taskinfo?taskID=21283491 ostree https://koji.fedoraproject.org/koji/taskinfo?taskID=21283698 gcc https://koji.fedoraproject.org/koji/taskinfo?taskID=21283588 checkpolicy https://koji.fedoraproject.org/koji/taskinfo?taskID=21283909
Some of these might be fixed in Rawhide; our repodata snapshot we use for depsolving and generating the module is (by definition, it's a snapshot) somewhat behind. We will be rebasing it soon but we can include individual fixes in the meantime.
Should we track our findings here? ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
dustymabe added a new comment to an issue you are following: ``
Should we track our findings here?
yes please
On another note, looking at the ostree rpm error I see:
``` error: Bad exit status from /var/tmp/rpm-tmp.Bvv1rE (%build) bogus date in %changelog: Sat Aug 25 2013 Colin Walters walters@verbum.org - 2013.5-3 bogus date in %changelog: Sat Aug 25 2013 Colin Walters walters@verbum.org - 2013.5-2 ```
are we building with a more strict version of the build tools in MBS vs normal koji? ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: ``
are we building with a more strict version of the build tools in MBS vs normal koji?
Nope. These warnings are unrelated to the build failing and you have them in your passing builds, too. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` Kicked off another build with @tdawson 's fixes. It will take a while to build & fail, of course. Feel free to watch the progress with `mbs-build watch 806`. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` Ok, failures from the last build:
* `selinux-policy` https://koji.fedoraproject.org/koji/taskinfo?taskID=21364166 -- should be fixable by updating `policycoreutils` to `2.7`; updating the package itself won't hurt either, so I'll look into this * `libtirpc` https://koji.fedoraproject.org/koji/taskinfo?taskID=21364167 -- bug reported (https://bugzilla.redhat.com/show_bug.cgi?id=1482063); I'll apply it and update our package * `kernel` https://koji.fedoraproject.org/koji/taskinfo?taskID=21364170 -- should be fixed in the latest kernels, I'll update it * `cockpit` https://koji.fedoraproject.org/koji/taskinfo?taskID=21364159 -- already fixed by updating to the yesterday's release (`149`) * `setools` https://koji.fedoraproject.org/koji/taskinfo?taskID=21364160 -- fixed in the latest version, fixed in our set as well
I need to leave early today and I'm traveling tomorrow so I'm not sure I can fix it all before the meeting but I'll try. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` I have included fixes for (hopefully) everything listed above and kicked off another build. Watch the progress with `mbs-build watch 812`.
If it goes well, what's the next steps? Should I be pursuing new compose configuration in Fedora Infra or should OSTree images be created elsewhere? ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
psabata added a new comment to an issue you are following: `` `atomic:master:20170822140830` built. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
walters added a new comment to an issue you are following: `` We generated an rpm-md repo: https://kojipkgs.stg.fedoraproject.org/compose/branched/psabata/Fedora-Modul...
I was able to compose a tree with https://pagure.io/fork/walters/fedora-atomic/c/dd3a727dfaaf491de5d24d1bb9e41... but at least it looks like `kernel-install` is failing in the root, probably need to make progress on https://github.com/projectatomic/rpm-ostree/pull/931 ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/312
atomic@lists.fedoraproject.org
-
Colin Walters -
Dusty Mabe
-
Petr Šabata