infrastructure November 2020

infrastructure@lists.fedoraproject.org

23 participants
20 discussions

VDO
by Miroslav Suchý 04 Jul '26

04 Jul '26

After I seen the talk about VDO: https://www.youtube.com/watch?v=7CGr5LEAfRY I went ahead and tried it. I tried it on small (12GB) sample of Copr data and I saved 20-30% data. I then deployed it on production server retrace.fedoraproject.org and saved there 15% out of 2TB. Several notes: The RPM packages are only available for RHEL. Recently the public github has been updated with version of VDO, which should work on Fedora. No RPM available for Fedora yet though. Here are my notes: # vdo create --name=vdo1 --device=/dev/vdb --vdoLogicalSize=1T # mkfs.ext4 /dev/mapper/vdo1 Some slides suggest to use -E discard. This is not needed for new volumes. See vdo-devel mailing list for reasoning. # mount -o discard /dev/mapper/vdo1 /mnt/a Probably best scenario is to create vdo on whole disk and then make PV from whole VDO disk and put LVM on top of that. When you already have data, you cannot convert your disk to VDO. You need create new VDO disk, format it, mount it and copy the data. In my case, due limited space for migration I created a small LVM volume "vdosrv": # lvcreate -L 80G -n vdosrv vol0 I converted it to VDO which claims to have 2TB of vitual total size # vdo create --name=srv18 --device=/dev/vol0/vdosrv --vdoLogicalSize=2T # mkfs.ext4 /dev/mapper/srv18 See how many space is *actually* free. # vdostats # mkdir /mnt/srv18 # mount -o discard /dev/mapper/srv18 /mnt/srv18 Now I moved some data from old volume to /mnt/srv18. I shrank the FS of old volume, I shrank the old LV. And then I enlarged the new LV. # lvresize -L +400G /dev/vol0/vdosrv and let know vdo that it can use the new space: # vdo growPhysical -n srv18 I repeated this several times. I have to say that my experience is so far good. I hope that my notes help someone. Miroslav

10 14

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

[PATCH 0/1] Add a flatpak-indexer openshift service
by Owen Taylor 01 Dec '20

01 Dec '20

From: "Owen W. Taylor" <otaylor(a)fishsoup.net> This is an initial attempt to create a configuration for flatpak-indexer to replace regindexer and add an image delta capability. The config here is derived from a working openshift configuration, but is untested in this form. See: https://pagure.io/fedora-infrastructure/issue/9272 Open questions: How to propagate content to the registry.fedoraproject.org reverse proxy ======================================================================== Currently the regindexer-generated content is rsync'ed from sundries to fedora-web/registry. How should this be done with flatpak-indexer running as an openshift app? Some possibilities that come to mind: - Run a rsyncd within the openshift app (either as a separate deploymentconfig or as a sidecar to the indexer) and expose a route to it internally in Fedora infrastructure. - Run a web server within the openshift app, expose a route to it internally in Fedora infrasturcture, and reverse proxy the content on fedora-web/registry instead of rsync'ing it. - Write the content onto a netapp volume, and mount that volume RO either on a host running rsyncd or directly on fedora-web/registry. What to use for a redis image ============================= Redis is used for caching and communication between the components. What redis image should be used? registry.redhat.io/rhel8/redis-5 needs configuration of a subscription docker.io/library/redis:5 centos/redis-5-centos7 don't rely on such images currently Custom Dockerfile image built from fedora:32 how would rebuilds be triggered? For the two other images needed here, I used ubi8 images - which aren't currently used elsewhere, but are presumably ok. How to handle identifying versions to build for staging/production ================================================================== I see that most openshift applications simply use 'staging'/'production' tags in the upstream repo, while a few take the approach of having specific hashes checked into the infrastructure ansible repository. Is the upstream tag approach considered sufficiently secure? (Making the service write a malicious index could allow causing users to upgrade to arbitrary application binaries.) Owen W. Taylor (1): Add a flatpak-indexer openshift service playbooks/openshift-apps/flatpak-indexer.yml | 56 +++++ .../reversepassproxy.registry-generic.conf | 34 ++- .../flatpak-indexer/files/imagestream.yml | 52 +++++ .../flatpak-indexer/files/service.yml | 16 ++ .../flatpak-indexer/files/storage.yml | 24 ++ .../flatpak-indexer/templates/buildconfig.yml | 84 +++++++ .../flatpak-indexer/templates/configmap.yml | 98 ++++++++ .../templates/deploymentconfig.yml | 221 ++++++++++++++++++ .../flatpak-indexer/templates/secret.yml | 11 + roles/regindexer/build/tasks/main.yml | 21 -- roles/regindexer/build/templates/config.yaml | 74 ------ 11 files changed, 584 insertions(+), 107 deletions(-) create mode 100644 playbooks/openshift-apps/flatpak-indexer.yml create mode 100644 roles/openshift-apps/flatpak-indexer/files/imagestream.yml create mode 100644 roles/openshift-apps/flatpak-indexer/files/service.yml create mode 100644 roles/openshift-apps/flatpak-indexer/files/storage.yml create mode 100644 roles/openshift-apps/flatpak-indexer/templates/buildconfig.yml create mode 100644 roles/openshift-apps/flatpak-indexer/templates/configmap.yml create mode 100644 roles/openshift-apps/flatpak-indexer/templates/deploymentconfig.yml create mode 100644 roles/openshift-apps/flatpak-indexer/templates/secret.yml delete mode 100644 roles/regindexer/build/tasks/main.yml delete mode 100644 roles/regindexer/build/templates/config.yaml -- 2.28.0

2 3

Is it possible delete my FAS account?
by Qian Cai 28 Nov '20

28 Nov '20

Hi there, in the process to delete my FAS account (caiqian), it looks like I can only mark it inactive. However, the docs said 'to remove all personal information, and mark the account as "Inactive."', so I "accidentally" also changed the email address to something totally random like xxx@xxx (I forgot the exactly). Later, I want to make sure that email is totally non-existent, so that nobody could access the FAS account anymore (it has some privileges). The bad news is that even though I can still remember the password, it now will ask me to enter the email address to reset the password since it was marked inactive. Since I don't remember the email and have no access to it, I can't reset it. Is there any way to remove the FAS account entirely?

2 3

Fedora packages site down
by Phil Clayton 27 Nov '20

27 Nov '20

For a while now https://apps.fedoraproject.org/packages/ has been reporting Service Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. Has this packages site been retired or does a server just need a boot? :)

8 14

Potential service degradation - containers and flatpaks builds (OSBS) - 2020-11-26 09:30 UTC
by David Kirwan 25 Nov '20

25 Nov '20

There will be potential service degradation for OSBS starting at 2020-11-26 09:30 UTC Work is being carried out to add an aarch64 build cluster to production OSBS. While this work is being carried out you may experience some containers or flatpaks build failures. Please follow the outage ticket https://pagure.io/fedora-infrastructure/issue/9489 to keep track of its status. Thanks, Fedora OSBS Initiative -- David Kirwan Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108 IM: @dkirwan

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 25 Nov '20

25 Nov '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-11-26 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

2 1

CPE Weekly: 2020-11-22
by Aoife Moloney 23 Nov '20

23 Nov '20

Hi Everyone, Below is this week's CPE weekly for week ending 2020-11-22 for both Fedora & CentOS, and if you want to visit the hackmd link https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view you can then use the header bar on your left to skip to Fedora or CentOS updates that interest you. ## General Project Updates Final project submission date for consideration in Q1 is Friday 27th November. If you have an initiative that may take weeks/months and multiple people to work on and want to request it to CPE, please follow the steps outlined in our initiatives repo and create your issue before 27th November https://pagure.io/cpe/initiatives-proposal Our updated initative timetable can be viewed here for 2021 https://docs.fedoraproject.org/en-US/cpe/time_tables/ Below are the projects the CPE team are currently working on for the months of October, November & December: * CentOS Stream Phase 4 - Build system services * Noggin Phase 4 - Data Migration of Fedora & CentOS Accounts, Community testing * OSBS for aarch64 - this will begin in November * Fedora Messaging Schemas - this work is continuing from Q3 and is being worked on part-time ### Misc #### GitLab New GitLab topic sent to devel-announce(a)lists.fedoraproject.org & centos-devel(a)centos.org on Message Bus is out. See email in hackmd here https://hackmd.io/oZrDwbSeSWO-l_X65A1ndg?view ## Project Updates *The below updates are pulled directly from our CPE team call we have every week.* ### Fedora ### Staging Environment * Completed - any issues you find please report them in fedora infra https://pagure.io/fedora-infrastructure/issues ### Noggin/AAA * Testing team owned apps in staging with Noggin * We will be requesting community member testing before December so keep an eye out for an announcement! * The teams kanban board where they track their work can be found here https://github.com/orgs/fedora-infra/projects/6 * And we have a project tracker available to be viewed here https://github.com/fedora-infra/aaa-tracker ### OSBS for aarch64 * Basic OKD 3.11 working on aarm64 with F31 * Working on repeating that install with F33 * Next step will be to ### Fedora Messaging Schemas * This project is worked on on a part time basis as we are prioritizing completing Noggin first before fully committing to its completion * There is a list of applications that require messaging schemas can be found here https://hackmd.io/@nilsph/H1i8CAbkP/edit * There is a readme which contains documentation on messaging schemas, a cookie-cutter template to create the schema and a definition of Done for writing a schemas https://github.com/fedora-infra/fedora-messaging-schemas-issues * The board they are working from can be viewed here https://github.com/orgs/fedora-infra/projects/7 ## CentOS Updates ### CentOS * CentOS 6 is EOL 30th November * CFP for FOSDEM Dojo - https://wiki.centos.org/Events/Dojo/FOSDEM2021 * Updated CentOS CI Openshift staging cluster to latest 4.6.4, Waiting for stable release in the 4.6 branch before rolling out to production. * CentOS 7.9.2009 is released! (for x86_64, i386, ppc64, ppc64le, armhfp and aarch64 architectures) * Lot of work done for Noggin/AAA ### CentOS Stream * Use centos-stream-release package to convert from CentOS 6 to CentOS Stream before 30th November * Working on integrating ODCS in Stream * Curating out t_functional suite https://github.com/centos/sig-core-t_functional * Refining our testing for finding issues at distro-level ## Team Info ### CPE Product Owner Office Hours IRC office hours are now once per month.Below are the logs from the most recent meetings and dates for the next ones. #### #fedora-meeting-1 * Next Meeting: 2020-12-17 @ 1300 UTC on #fedora-meeting-1 #### #centos-meeting * Next Meeting: 2020-12-15 @ 1500 UTC on #centos-meeting ## Background: The Community Platform Engineering group, or CPE for short, is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. See our wiki page here for more information:https://docs.fedoraproject.org/en-US/cpe/ As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great week! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 18 Nov '20

18 Nov '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-11-19 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

ancient backups of the moinmoin wiki?
by Matthew Miller 17 Nov '20

17 Nov '20

This is a long shot, I know. When we converted to MediaWiki from MoinMoin in 2008, pages were imported, but history was lost. I often like to go spelunking in the past, and it's often frustrating to hit this barrier. As I recall, MoinMoin stored everything in flat files, so it wouldn't even really need to be stood up again to get a look at the history. If someone happened to keep it, that is. Is there any chance we have a tarball snapshot of the wiki circa 2008? -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

5 5

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure November 2020