copr-devel October 2022

copr-devel@lists.fedorahosted.org

6 participants
7 discussions

Mock/Copr default epel-8-* configuration to be changed
by Pavel Raiskup 29 Jul '25

29 Jul '25

Hello Fedora EPEL maintainers! First I don't feel comfortable announcing this, I'm not happy about the situation and so I don't want to be the lightning rod :-). But I believe that we can come to acceptable Copr/Mock solution and this needs to be discussed... so here we are. By the end of the year 2021 we have to fix our default EPEL 8 Mock configuration (mock-core-configs.rpm, /etc/mock/epel-8-*.cfg) as CentOS 8 goes EOL by then. The same thing needs to happen in Fedora Copr, with the epel-8-* chroots (side note, in Fedora Copr we use the mock-core-configs package for builds without any deployment specific modifications). I am proposing (as PR against mock upstream ATM [1]) to switch the default epel-* configuration from CentOS+EPEL to RHEL+EPEL as soon as possible (see the pull request [1]). This would bring some consequences, namely newly with epel-8* chroots, - builds will require a valid Red Hat subscription (the no-cost variant is OK as well, though [2]) - we will miss some build-time packages that Red Hat is not shipping, at least at the beginning till they are added (to RHEL CRB, or other currently unknown place). - cross-arch compilation can not be used, Red Hat subscriptions don't allow that (using QEMU and rpm --forcearch), [3] The positive thing is that the default configuration will be much closer to the official EPEL builds (because Fedora Koji EPEL builds are actually done also against RHEL). For the Fedora Copr builders, we already have the necessary Red Hat subscriptions in hand (will be deployed by the end of the year). So we will only loose the opportunity to build on emulated epel-8-armhfp permanently, and epel-8-s390x temporarily (as we already work on the native s390x support). Any thoughts? Feedback is needed here. [1] https://github.com/rpm-software-management/mock/pull/802 [2] https://rpm-software-management.github.io/mock/Feature-rhelchroots [3] https://bugzilla.redhat.com/show_bug.cgi?id=1912847 Pavel

11 22

Oracle Linux chroots (temporarily) available in Fedora Copr
by Pavel Raiskup 17 Jul '23

17 Jul '23

Hello, just a quick note - we enabled Oracle Linux chroots in Fedora Copr today, this is temporarily needed by OAMG/LEAPP team for their CI/CD system. Feel free to test those chroots, but please don't heavily depend on them as we don't know when we'll again drop them (preferably use epel/centos-stream chroots instead, they should provide similar build results). Pavel

2 2

Excluding older builds of packages from Fedora when testing new ones in Copr
by Miro Hrončok 24 May '23

24 May '23

Hello Copr users and developers. When we update packages in Fedora, we regularly use Copr to test the impact of the upgrade. For me, the procedure usually goes like this: 1) create a new copr with Fedora rawhide x86_64 chroot (and added Koji repo) $ copr create $COPR --repo='http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/' --chroot fedora-rawhide-x86_64 --delete-after-days 30 2) define and build the updated package in the copr $ copr add-package-distgit $COPR --name $PKG --webhook-rebuild on --commit $BRANCH --namespace forks/$(whoami) $ copr build-package $COPR --name $PKG 3) get the list of dependent packages $ repoquery -q --repo=rawhide{,-source} [--whatrequires $spkg for each subpackage] --recursive | grep src$ | pkgname 4) define and build the depended packages in the copr $ parallel copr add-package-distgit $COPR --webhook-rebuild on --commit rawhide --name -- $(repoquery from above ...) $ parallel copr build-package $COPR --nowait --background --name -- $(repoquery from above ...) 5) analyze build failures, do a "control" rebuild in another copr if needed However, this procedure has a flaw. Let's say I'm working on upgrading python-click from 7.x to 8.x. And let's say a package (even transitively) BuildRequires: python3dist(click) < 8 The way that dnf dependency resolution works, that package will be built with Rawhide's python3-click 7.x and it will be marked as successful. However, I'd like to see a failure here to be notified that such package cannot be build and will be negatively impacted by the update. Is there a way to solve this? I have couple ideas, but none of them is fully working: A) Compose my own repo with the updated package and Rawhide content without it, use that repo in the copr. Pros: - this is similar to what will happen in Koji once the package is updated Cons: - this requires tooling that I don't think exists - this requires a place to put that repo to - the repo creation could take a lot of time and would need to be repeated on-demand each time rawhide changes - Copr's Fedora chroots always include Fedora repos (maybe I can use custom-1-x86_64 chroot?) B) Create a Fedora side tag, explicitly block the package from it, use that side tag's Koji repo. Pros: - same as (A) Cons: - I don't think on-demand side tags allow users to block packages - Copr's Fedora chroots always include Fedora repos (same as (A)) - this wastes Koji's resources a bit - requires waiting for the initial Koji regen-repo C) Block (exclude) python3-click < 8 from the chroot. Pros: - no custom repos required - no resources overhead - no time overhead Cons: - There is no way exclude packages in chroot settings. Mock settings possibly allow me to do this in config_opts['dnf.conf']. - The exclude could obfuscate root.log resolution problems logs. - Packager needs to figure out what exactly to exclude (possibly need to exclude all subpackage's NEVRAs from rawhide compose (and Koji buildroot if they differ)) Is there another way? If not, I think (C) is easiest to actually implement, if the chroot settings page in copr gains an "excludes" option. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 7

Outage: Upgrading the Fedora Copr backend storage, 2022-10-28 -- 2022-10-30
by Pavel Raiskup 31 Oct '22

31 Oct '22

We need to replace the volume hosting our RPM/YUM repo data with a larger one. As a consequence, we'll have to stop the build queue processing for about 20 hours, and we'll have two "full" (but short) outages (2x5 minutes) when the yum repositories will be unavailable. The anticipated start of the outage (queue stops processing) is Saturday 2022-10-29: $ date --date "2022-10-29 17:00 UTC" Sat Oct 29 07:00:00 PM CEST 2022 But a few hours before, we'll lower the build throughput a bit to prioritize the I/O we have on the data migration task.. We'd like to kindly ask you to minimize the build throughput in the Fedora Copr build service during the weekend in question — just to allow us to make the transition as smooth as possible. It would be ideal if you could e.g. delay your mass rebuilding and other large data modifications tasks until the storage movement is over. That said, we don't plan to turn the frontend machine off (API and web-UI) during the whole outage period. For more info, see the more detailed schedule: https://docs.google.com/spreadsheets/d/1URopuw2R533H6i4vSTH8Nt47EIhs3ubWcLU… Affected services: - Copr service (frontend): https://copr.fedorainfracloud.org - Copr (backend) repositories: https://download.copr.fedorainfracloud.org/ (AWS CloudFront proxy) https://copr-be.cloud.fedoraproject.org/ (storage) Sorry for the inconvenience, Copr Team

4 5

reporting a a very old bug
by Sérgio Basto 24 Oct '22

24 Oct '22

Hi, The bug is simply to report, I found it some months ago it with help of FrostyX IIRC, after install my own corp build in a laptop . The bug is after web page of results show the firsts results, the web page is cached and is never updated again. While new files of log are accessible the web page doesn't show it , this happens to me a lot when I'm looking at builder-live.log.gz file while building is happening . For example https://download.copr.fedorainfracloud.org/results/sergiomb/vboxfor23/fedor… doesn't show any build.log.gz and build.log.gz is there in the same place ase usual https://download.copr.fedorainfracloud.org/results/sergiomb/vboxfor23/fedor… Thanks , -- Sérgio M. B.

5 13

ATTENTION: Copr will be temporarily deleting failed and outdated builds after 7 days
by Jiri Kyjovsky 21 Oct '22

21 Oct '22

Hello, Due to problems with free disk storage, Copr will be temporarily deleting failed and outdated builds after 7 days instead of the default 14 days until migration to another disk storage takes place. Best regards Copr Team

4 3

OUTAGE: Fedora Copr enlarging the backend infra VM
by Pavel Raiskup 17 Oct '22

17 Oct '22

We plan to power-cycle one virtual machine in our infrastructure and boost the computational power there. In thurn the build results (RPMs/metadata) are going to be offline for a short period of time (up to 10 minutes). More info on #fedora-buildsys irc channel (libera chat). Links: https://pagure.io/fedora-infrastructure/issue/10943 https://status.fedoraproject.org/ Pavel

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

copr-devel October 2022