infrastructure January 2022

infrastructure@lists.fedoraproject.org

18 participants
18 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

hyperkitty 1.3.5
by Reon Beon 05 Apr '22

05 Apr '22

Getting closer now.

1 2

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 31 Jan '22

31 Jan '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-02-01 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

CPE Weekly Update - Week of January 24th - 28th
by Patrik Polakovic 30 Jan '22

30 Jan '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/) If you wish to read this in the form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-january-2… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * Found we had users with _ . - in usernames and were allowing one character. Fixed noggin and notified all users affected to make new accounts. Will close invalid ones at end of month. * Got 3 new x86 machines added to openqa for additional capacity. * Got 5 new/old x86 machines ready to add to ocp4 clusters. * Tracked down a bodhi bug with adamw and it got fixed and fix deployed! * Merged a bunch of infra/ansible PR’s that had been waiting. * Found that notifs-backend01 is the last thing keeping fas2 around. :( ### CentOS Infra including CentOS CI * SIGs can now push their centos-release* pkgs to mirror.stream.centos.org for public consumption. * Big storage reorganization (multiple HDD issues, or pending - pfa mode-) , going from out of warranty to still out of warranty node (risk mitigation). * Setting up two new IBM Power 9 (ppc64le) boxes for Stream 9 , SIGs and CI testing. * WIP: new infra for debuginfod service for Stream 8 and 9. * Business as usual: -- New cbs/koji tags and modifying some (welcome Tomas Hrcka to the CentOS infra team and first ticket assigned to him: adding EPEL{-next} for buildroots). -- Some new mirrors for Stream 9. ### Release Engineering * Mass rebuild finished, then we did another pass, and there’s going to be yet one more when latest gcc finishes building. ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * MR is mostly ready to light up the Stream 8 koji tag structure in the consolidated buildsystem. * Preparing for module branching . * Stream 9 Buildsystem slated for a cage move on 11-Mar-2022. ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Ongoing: -- Legacy API. -- Node pool maintenance. ## Image builder for Fedora IoT Goal of this Initiative ----------------------- Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra. Updates ------- * The team is focusing on getting all components installed locally: -- Osbuild-composer -- Koji + plugins * Hopefully will be in a good place to talk to image builder from koji stg once we know what setting up the plugins looks like. ## Bodhi Goal of this Initiative ----------------------- This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management and automate part of the release process. Read ARC team findings in detail at: https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html Updates ------- * Version 5.7.4 released. ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL9 up to 1539 source packages (increase of 193 from last week). * Several new packagers have expressed specific interest in EPEL. * Three talks related to EPEL happening next week at the February CentOS Dojo. Kindest regards, CPE Team

1 0

RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
by Roberto Sassu 26 Jan '22

26 Jan '22

Hi everyone (note for the infrastructure mailing list: please check if the changes I'm proposing could be tested in the Fedora infrastructure, like Copr) I made the first version of the rpm extension to sign fsverity digests with a GPG key. The patch set (with some bug fixes) is available here: https://github.com/robertosassu/rpm/commits/fsverity-gpg-v1 I tested it locally with my own GPG key. I took an existing Fedora 34 package and signed it with rpmsign: $ usr/bin/rpmsign --define "%_gpg_name testhost <testhost(a)test.test>" \ --define "%_file_signing_key _GPG_" \ --define "%_file_signing_cert _GPG_" \ --addsign --signverity tmux-3.1c-2.fc34.x86_64.rpm I then checked that the package has now fsverity signatures: $ usr/bin/rpm -qp tmux-3.1c-2.fc34.x86_64.rpm \ --queryformat '[%{RPMTAG_FILENAMES} %{RPMTAG_VERITYSIGNATURES}\n]' [...] /usr/bin/tmux iQHHBAABCgAxFiEEEiFa0dGZVYzTrIN+rxtXRMfK0McFAmHq0+4THHRlc3Rob3N0 QHRlc3QudGVzdAAKCRCvG1dEx8rQx81nC/42NW9xJx3rcTiR6/5oL55GPkan+OIq t2dW1clJUOrxOGVy/5JQTQf0MQXA7gzH1yPgcrskkahjSfWlp4pt7oOw3rukUyaO zVZxue4XE6XESYtolczK4VEhpc8lbm4hj0e4NCg/dKri/+L5wIdJvmqWNeCfl7uZ [...] In a VM I tried to install the modified package. The root filesystem is ext4 and has the fsverity feature enabled. The fsverity rpm plugin is also enabled and hasn't been modified to work with the new PGP signatures. The kernel includes the patch set I recently sent to the kernel mailing lists to add support for PGP keys and signatures: https://lore.kernel.org/linux-integrity/20220111180318.591029-1-roberto.sas… and another patch that calls verify_pgp_signature() in fs/verity/signature.c. The first installation attempt fails, due to the missing key in the .fs-verity keyring: # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug [...] D: Plugin: calling hook fsm_file_prepare in fsverity plugin D: applying signature: [...] D: failed to enable verity (errno 126) for /usr/bin/tmux;61ead62d Then, I added the required GPG key to the .fs-verity keyring: # cat /mnt/repos/linux/certs/pubring.gpg | keyctl padd asymmetric test %keyring:.fs-verity 76292211 The key is now loaded: # keyctl show %keyring:.fs-verity Keyring 66741466 --a-swrv 0 0 keyring: .fs-verity 76292211 --als--v 0 0 \_ asymmetric: test I retried the tmux installation: # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug [...] D: Plugin: calling hook fsm_file_prepare in fsverity plugin D: applying signature: [...] D: fsverity enabled signature for: path /usr/bin/tmux;61ead713 dest /usr/bin/tmux This time the installation is successful, which means that the PGP signature has been successfully verified. Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua

3 3

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 24 Jan '22

24 Jan '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-01-25 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

CPE Feedback Survey Q3-Q4 2021
by Ant Carroll 24 Jan '22

24 Jan '22

Hey folks, We have been doing the Community Platform Engineering feedback survey for a while now, and we still use the feedback collected to adjust our communications and processes. We would like to hear your thoughts again through a short survey we've put together to learn how your experiences have been with us since July 2021. If you could take the time (5mins max) to complete it, it would be hugely valuable as we work on this continuous improvement - https://fedoraproject.limequery.com/6 The survey will remain open until Feb 07th (23:59 UTC). Know more about the team at: https://docs.fedoraproject.org/en-US/cpe/ Cheers, Ant -- Ant Carroll Associate Manager, Software Engineering Red Hat Waterford <http://www.redhat.com> Communications House Cork Road, Waterford City ancarrol(a)redhat.com M: +353876213163 IM: ancarrol @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://www.redhat.com/>

1 0

Unsuscribe to list
by Patrice Dufourt 23 Jan '22

23 Jan '22

Le sam. 22 janv. 2022, 05:04, < infrastructure-request(a)lists.fedoraproject.org> a écrit : > Send infrastructure mailing list submissions to > infrastructure(a)lists.fedoraproject.org > > To subscribe or unsubscribe via email, send a message with subject or > body 'help' to > infrastructure-request(a)lists.fedoraproject.org > > You can reach the person managing the list at > infrastructure-owner(a)lists.fedoraproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of infrastructure digest..." > > Today's Topics: > > 1. CPE Weekly Update - Week of January 17th-22nd (Vipul Siddharth) > 2. RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal) > (Roberto Sassu) > > > ---------------------------------------------------------------------- > > Date: Fri, 21 Jan 2022 19:46:41 +0530 > From: Vipul Siddharth <siddharthvipul1(a)gmail.com> > Subject: CPE Weekly Update - Week of January 17th-22nd > To: Fedora Infrastructure <infrastructure(a)lists.fedoraproject.org>, > centos-devel(a)centos.org, Development discussions related to > Fedora > <devel(a)lists.fedoraproject.org> > Message-ID: > < > CA+Bo6C0h4C8hoLXikcsQdhEywEcM5FmpRkwo0tiryqz2jbM6bg(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi everyone, > > This is a weekly report from the CPE (Community Platform Engineering) > Team. If you have any questions or feedback, please respond to this > report or contact us on #redhat-cpe channel on libera.chat > (https://libera.chat/) > > We (CPE team) will be joining Fedora Social Hour on Jan 27th. > Looking forward to seeing a lot of you! > ( > https://discussion.fedoraproject.org/t/join-us-for-fedora-social-hour-every… > ) > > If you wish to read this in form of a blog post, check the post on > Fedora community blog: > ( > https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-january-1… > ) > > > # Highlights of the week > > ## Infrastructure & Release Engineering > Goal of this initiative > ----------------------- > Purpose of this team is to take care of day to day business regarding > CentOS and Fedora Infrastructure and Fedora release engineering work. > It’s responsible for services running in Fedora and CentOS > infrastructure and preparing things for the new Fedora release > (mirrors, mass branching, new namespaces etc.). The ARC (which is a > subset of the team) investigates possible initiatives that CPE might > take on. > > Update > ------ > > ### Fedora Infra > * All koji builders/hubs upgraded to F35 and ready for mass rebuild ( 🤞 ) > * Additional s390x disk space appeared, so added 10 more s390x builders. > * Fixed IPA issue with certs ( known upgrade bug) > * Difficult container builds failing issue solved. > > > ### CentOS Infra including CentOS CI > * CentOS Linux 8 EOL plan > * Hardware issues (storage box, 64 compute nodes for CI infra) > * Kmods SIG DuD discussion (koji plugin vs external script) > * CI storage for ocp/openshift migration completed and working > faster/better ! > * CentOS CI tenants Survey (for the upcoming DC move) > > > > ### Release Engineering > * Mass rebuild starts today > * Several rawhide issues fixed and composes have been good the last few > days. > > > > ## CentOS Stream > Goal of this initiative > ----------------------- > This initiative is working on CentOS Stream/Emerging RHEL to make this > new distribution a reality. The goal of this initiative is to prepare > the ecosystem for the new CentOS Stream. > > Updates > ------- > * The NFV repo was added to CentOS Stream 8, work is happening now on > the repo files in centos-release > * Module branching work is ongoing > * Libffi is causing some interesting breakage in ELN > * GCC bugs in ELN > * Koji/brew Inheritance discussions are still happening > * Testing Content Resolver with production data before deployment > > > > > ## Datanommer/Datagrepper V.2 > Goal of this initiative > ----------------------- > The datanommer and datagrepper stacks are currently relying on fedmsg which > we want to deprecate. > These two applications need to be ported off fedmsg to fedora-messaging. > As these applications are 'old-timers' in the fedora infrastructure, we > would > also like to look at optimizing the database or potentially redesigning it > to > better suit the current infrastructure needs. > For a phase two, we would like to focus on a DB overhaul. > > Updates > ------- > * It’s done! Data is migrated, the new code is now running in prod. > > > ## CentOS Duffy CI > Goal of this initiative > ----------------------- > Duffy is a system within CentOS CI Infra which allows tenants to provision > and > access bare metal resources of multiple architectures for the purposes of > CI testing. > We need to add the ability to checkout VMs in CentOS CI in Duffy. We have > OpenNebula hypervisor available, and have started developing playbooks > which > can be used to create VMs using the OpenNebula API, but due to the current > state > of how Duffy is deployed, we are blocked with new dev work to add the > VM checkout functionality. > > Updates > ------- > * Legacy API > * Node Pools & Ansible Backend > > > ## Image builder for Fedora IoT > Goal of this initiative > ----------------------- > Integration of Image builder as a service with Fedora infra to allow > Fedora IoT > migrate their pipeline to Fedora infra. > > Updates > ------- > * Officially kicked off this week > * Fact finding at the moment > * Met with Peter Robinson and team from Fedora IoT > * Need to figure out a way to run their pipeline > * At least 1 koji plugin to be written + deployed > * Meeting with Image Builder team tomorrow > * They are currently blocked by auth, development underway > * Need to get an idea of their API and what they expect from us > > > ## Bodhi > Goal of this initiative > ----------------------- > This initiative is to separate Bodhi into multiple sub packages, > fix integration and unit tests in CI, fix dependency management, > and automate part of the release process. > Read ARC team findings in detail at: > https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html > > Updates > ------- > * splitting the codebase into separate python packages > * migrating from CentOS CI to Zuul > > > ## EPEL > Goal of this initiative > ----------------------- > Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest > Group that creates, maintains, and manages a high quality set of additional > packages for Enterprise Linux, including, but not limited to, > Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), > Oracle Linux (OL). > > EPEL packages are usually based on their Fedora counterparts and will never > conflict with or replace packages in the base Enterprise Linux > distributions. > EPEL uses much of the same infrastructure as Fedora, including buildsystem, > bugzilla instance, updates manager, mirror manager and more. > > Updates > ------- > * epel9 up to 1346 source packages (increase of 188 from last week) > * Two talks submitted and accepted for the February CentOS Dojo > * State of EPEL > * EPEL Packaging Hackfest > > > > Kindest regards, > -- > Vipul Siddharth > He/His/Him > On behalf of the CPE team > > ------------------------------ > > Date: Fri, 21 Jan 2022 16:08:04 +0000 > From: Roberto Sassu <roberto.sassu(a)huawei.com> > Subject: RE: F36 Change: Enable fs-verity in RPM (System-Wide Change > proposal) > To: Development discussions related to Fedora > <devel(a)lists.fedoraproject.org> > Cc: "infrastructure(a)lists.fedoraproject.org" > <infrastructure(a)lists.fedoraproject.org> > Message-ID: <357929364e1944958ade6098401ef2ce(a)huawei.com> > Content-Type: text/plain; charset="utf-8" > > Hi everyone > > (note for the infrastructure mailing list: please check if the changes > I'm proposing could be tested in the Fedora infrastructure, like Copr) > > I made the first version of the rpm extension to sign fsverity > digests with a GPG key. The patch set (with some bug fixes) > is available here: > > https://github.com/robertosassu/rpm/commits/fsverity-gpg-v1 > > I tested it locally with my own GPG key. I took an existing > Fedora 34 package and signed it with rpmsign: > > $ usr/bin/rpmsign --define "%_gpg_name testhost <testhost(a)test.test>" \ > --define "%_file_signing_key _GPG_" \ > --define "%_file_signing_cert _GPG_" \ > --addsign --signverity > tmux-3.1c-2.fc34.x86_64.rpm > > > I then checked that the package has now fsverity signatures: > > $ usr/bin/rpm -qp tmux-3.1c-2.fc34.x86_64.rpm \ > --queryformat '[%{RPMTAG_FILENAMES} > %{RPMTAG_VERITYSIGNATURES}\n]' > [...] > /usr/bin/tmux > iQHHBAABCgAxFiEEEiFa0dGZVYzTrIN+rxtXRMfK0McFAmHq0+4THHRlc3Rob3N0 > QHRlc3QudGVzdAAKCRCvG1dEx8rQx81nC/42NW9xJx3rcTiR6/5oL55GPkan+OIq > t2dW1clJUOrxOGVy/5JQTQf0MQXA7gzH1yPgcrskkahjSfWlp4pt7oOw3rukUyaO > zVZxue4XE6XESYtolczK4VEhpc8lbm4hj0e4NCg/dKri/+L5wIdJvmqWNeCfl7uZ > [...] > > In a VM I tried to install the modified package. The root filesystem > is ext4 and has the fsverity feature enabled. > > The fsverity rpm plugin is also enabled and hasn't been modified > to work with the new PGP signatures. > > The kernel includes the patch set I recently sent to the kernel > mailing lists to add support for PGP keys and signatures: > > > https://lore.kernel.org/linux-integrity/20220111180318.591029-1-roberto.sas… > > and another patch that calls verify_pgp_signature() in > fs/verity/signature.c. > > The first installation attempt fails, due to the missing key > in the .fs-verity keyring: > > # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug > [...] > D: Plugin: calling hook fsm_file_prepare in fsverity plugin > D: applying signature: > [...] > D: failed to enable verity (errno 126) for /usr/bin/tmux;61ead62d > > Then, I added the required GPG key to the .fs-verity keyring: > > # cat /mnt/repos/linux/certs/pubring.gpg | keyctl padd asymmetric test > %keyring:.fs-verity > 76292211 > > The key is now loaded: > > # keyctl show %keyring:.fs-verity > Keyring > 66741466 --a-swrv 0 0 keyring: .fs-verity > 76292211 --als--v 0 0 \_ asymmetric: test > > I retried the tmux installation: > > # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug > [...] > D: Plugin: calling hook fsm_file_prepare in fsverity plugin > D: applying signature: > [...] > D: fsverity enabled signature for: path /usr/bin/tmux;61ead713 dest > /usr/bin/tmux > > This time the installation is successful, which means that the PGP > signature has been successfully verified. > > Roberto > > HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 > Managing Director: Li Peng, Zhong Ronghua > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapr… > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > > > ------------------------------ > > End of infrastructure Digest, Vol 187, Issue 10 > *********************************************** >

1 0

CPE Weekly Update - Week of January 17th-22nd
by Vipul Siddharth 21 Jan '22

21 Jan '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/) We (CPE team) will be joining Fedora Social Hour on Jan 27th. Looking forward to seeing a lot of you! (https://discussion.fedoraproject.org/t/join-us-for-fedora-social-hour-every…) If you wish to read this in form of a blog post, check the post on Fedora community blog: (https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-january-1…) # Highlights of the week ## Infrastructure & Release Engineering Goal of this initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * All koji builders/hubs upgraded to F35 and ready for mass rebuild ( 🤞 ) * Additional s390x disk space appeared, so added 10 more s390x builders. * Fixed IPA issue with certs ( known upgrade bug) * Difficult container builds failing issue solved. ### CentOS Infra including CentOS CI * CentOS Linux 8 EOL plan * Hardware issues (storage box, 64 compute nodes for CI infra) * Kmods SIG DuD discussion (koji plugin vs external script) * CI storage for ocp/openshift migration completed and working faster/better ! * CentOS CI tenants Survey (for the upcoming DC move) ### Release Engineering * Mass rebuild starts today * Several rawhide issues fixed and composes have been good the last few days. ## CentOS Stream Goal of this initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * The NFV repo was added to CentOS Stream 8, work is happening now on the repo files in centos-release * Module branching work is ongoing * Libffi is causing some interesting breakage in ELN * GCC bugs in ELN * Koji/brew Inheritance discussions are still happening * Testing Content Resolver with production data before deployment ## Datanommer/Datagrepper V.2 Goal of this initiative ----------------------- The datanommer and datagrepper stacks are currently relying on fedmsg which we want to deprecate. These two applications need to be ported off fedmsg to fedora-messaging. As these applications are 'old-timers' in the fedora infrastructure, we would also like to look at optimizing the database or potentially redesigning it to better suit the current infrastructure needs. For a phase two, we would like to focus on a DB overhaul. Updates ------- * It’s done! Data is migrated, the new code is now running in prod. ## CentOS Duffy CI Goal of this initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Legacy API * Node Pools & Ansible Backend ## Image builder for Fedora IoT Goal of this initiative ----------------------- Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra. Updates ------- * Officially kicked off this week * Fact finding at the moment * Met with Peter Robinson and team from Fedora IoT * Need to figure out a way to run their pipeline * At least 1 koji plugin to be written + deployed * Meeting with Image Builder team tomorrow * They are currently blocked by auth, development underway * Need to get an idea of their API and what they expect from us ## Bodhi Goal of this initiative ----------------------- This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management, and automate part of the release process. Read ARC team findings in detail at: https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html Updates ------- * splitting the codebase into separate python packages * migrating from CentOS CI to Zuul ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * epel9 up to 1346 source packages (increase of 188 from last week) * Two talks submitted and accepted for the February CentOS Dojo * State of EPEL * EPEL Packaging Hackfest Kindest regards, -- Vipul Siddharth He/His/Him On behalf of the CPE team

1 0

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 17 Jan '22

17 Jan '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-01-18 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

koji hubs and builders updated
by Kevin Fenzi 15 Jan '22

15 Jan '22

Just thought I would share that (finally) we have completed upgrades on the koji hubs and builders. The hubs have been upgraded to Fedora 35 and the latest koji version (1.17.1). Builder hypervisors have been upgraded to RHEL8.5 and the latest advanced virt stack. Builders have been upgraded/reimaged to Fedora 35 and the latest packages. On armv7 builders: As you all know due to https://bugzilla.redhat.com/show_bug.cgi?id=1920183 ( Overeager OOM kills on 5.10 kernels on 32bit arm with lpae ) we had been running an old kernel on them, but after some testing in staging it seemed that the upcoming 5.16 kernel worked pretty well. So, All of them are running that now. There have been some OOM kills already, I'm playing around with options on them to try and get that number to 0. Hopefully they will be ok for the mass rebuild. On s390x builders: We got some more resources, so the kvm builders now have more memory and cpus. I've asked for some more disk space and once we have that I plan to spread those resources out to have 2x as many builders (ie, now: 4 cpus, 20gb mem x 10, after: 2 cpus, 10gb mem x 20) I don't know if this will be done in time for the mass rebuild, but hopefully so. On ppc64le: We have some more power9 hardware waiting to be put in service, so we should be adding some builders here, and/or spreading out resources. Thanks for everyone's patience with builds that have been taking a while or restarting. Do continue to report them and we will do the best we can to fix things. kevin

1 0

CPE Weekly Update – Week of January 10th – 14th
by Lenka Segura 14 Jan '22

14 Jan '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/) We (CPE team) will be joining Fedora Social Hour on Jan 27th. Looking forward to seeing a lot of you! ( https://discussion.fedoraproject.org/t/join-us-for-fedora-social-hour-every… ) If you wish to read this in the form of a blog post, check the post on Fedora community blog: (https://communityblog.fedoraproject.org/?p=10633) # Highlights of the week ## Infrastructure & Release Engineering Goal of this initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * Mostly quiet holidays with only minor reboots, etc * More koji upgrades: all aarch64/armv7 done, s390x kvm and hubs left to do * Container builds broken, needs more eyes: https://pagure.io/fedora-infrastructure/issue/10437 * Centos cert fetching broken, needs more eyes ### CentOS Infra including CentOS CI * CentOS Linux 8 EOL plan * Hardware issues (storage box, 64 compute nodes for CI infra) * Kmods SIG DuD discussion (koji plugin vs external script) * CI storage for ocp/openshift migration planning ### Release Engineering * Rawhide compose issues, but we got a good compose yesterday after a bunch of work * Mass rebuild of F36 next week ## CentOS Stream Goal of this initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Finished our January planning, working on: * Preparing new version of Content Resolver for production, finishing up stuff around the buildroot integration * Exploring things around increasing compose quality * Business as usual ## Datanommer/Datagrepper V.2 Goal of this initiative ----------------------- The datanommer and datagrepper stacks are currently relying on fedmsg which we want to deprecate. These two applications need to be ported off fedmsg to fedora-messaging. As these applications are 'old-timers' in the Fedora infrastructure, we would also like to look at optimizing the database or potentially redesigning it to better suit the current infrastructure needs. For phase two, we would like to focus on a DB overhaul. Updates ------- * Data is migrated, we need to deploy the new code to production now. ## CentOS Duffy CI Goal of this initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Work on backend -> modules to provision vms * Legacy API integration ## Image builder for Fedora IoT Goal of this initiative ----------------------- Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra. Updates ------- * Team forming this week. Currently waiting on work from the Image Builder team to wrap to unblock us from moving forward ## Bodhi Goal of this initiative ----------------------- This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management, and automate part of the release process. Read ARC team findings in detail at: https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html Updates ------- * Team is forming this week and will officially be launching work next Monday ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * epel9 is growing rapidly: * 2589 packages available (355 more in testing) * 1158 source rpms (225 more in testing) * Positive community response * Ongoing documentation improvements Kindest regards, CPE Team

1 0

Planned Outage - Datanommer Migration - 2022-01-17 11:00 UTC
by Mark O'Brien 14 Jan '22

14 Jan '22

Planned Outage - Datanommer Migration - 2022-01-17 11:00 UTC There will be an outage starting at 2022-01-17 11:00 UTC, which will last approximately 2 hours. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d '2022-01-17 11:00 UTC' Reason for outage: We are making some improvements to the performance of the datanommer database including adding the timescaledb plugin, a migration to a new database was required as this involved some breaking changes, the migration has already taken place but the required apps will now be required to point to the new database Affected Services: datanommer/datagrepper and any service which interacts with those Ticket Link: https://pagure.io/fedora-infrastructure/issue/10476 Please join #fedora-admin or #fedora-noc on irc.libera.chat or add comments to the ticket for this outage above. Mark

1 0

Datanommer / Datagrepper migration
by Aurelien Bompard 12 Jan '22

12 Jan '22

Hey folks! I am happy to report that the datanommer data migration is finally complete! \o/ Now we can move on to migrating the apps themselves. I had initially written this plan: https://github.com/fedora-infra/datanommer/wiki/Migration-plan We'll need to set a downtime window, I'd say of a couple hours, and switch to the new prod apps. Ideally I'd like to have a sysadmin with me during that window in case something goes wrong (even though it never happens, right?). I think Mark is back from vacation this Wednesday, shall we aim for a downtime next week or is it going to be bad timing with another work that's being done at the same time? Cheers Aurélien

4 6

automating SCM requests (again)
by Matthew Miller 11 Jan '22

11 Jan '22

Okay, this is making me _very_ sure that we should automate these: https://pagure.io/fedora-infrastructure/issue/10441 That's because even though a human is spending time running the script, there does not seem to be any opportunity for human intervention... this happened: https://pagure.io/releng/fedora-scm-requests/issue/39869 -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

5 4

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 10 Jan '22

10 Jan '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-01-11 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

Is anyone using greenwave 'decision update' fedora-messaging messages?
by Adam Williamson 06 Jan '22

06 Jan '22

Hi folks! So, greenwave publishes these 'decision.update' messages: https://apps.fedoraproject.org/datagrepper/id?id=2022-04ed73bc-25ca-4dfc-95… Up until relatively recently, they were used by Bodhi to update gating status for updates. But I figured that was a bad design for various reasons, and changed Bodhi to do it differently[0]. I'm not aware of anything else that uses these messages, I think they're bad design, and I'd like to submit a greenwave PR to remove all the code that's used to produce them (it would substantially simplify greenwave). But before doing that, it seems a good idea to ask if anyone's aware of anything else that uses them. I believe Kevin told me no native fedora-messaging consumers seem to be subscribed to that topic, but it is possible something could be consuming them via the fedmsg bridge. Thanks folks! [0] https://github.com/fedora-infra/bodhi/pull/4230 -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net

1 0

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 03 Jan '22

03 Jan '22

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2022-01-04 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure January 2022