infrastructure February 2021

infrastructure@lists.fedoraproject.org

14 participants
19 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Fedora 34 Beta Freeze now in effect
by Kevin Fenzi 28 Mar '21

28 Mar '21

Greetings. We are now in the infrastructure freeze leading up to the Fedora 34 Beta release. This is a pre release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 34 Beta when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git ansible/scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2021-03-16 (or later if release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, Kevin

3 2

Introducing the ARC sub-team in CPE - and first research topic
by Pierre-Yves Chibon 26 Feb '21

26 Feb '21

Good Morning Everyone, While planning work, the CPE team has realized that a number of our initiatives actually start with a research phase to find the most appropriate technical solution. This leads to some issues with planning as without knowing the technical solution we want to take, it's hard to evaluate the amount of work needed and thus the time it'll take to do it. In order to help with this, we're creating a small sub-team in CPE, called the ARC team for Advance Reconaissance Crew*. The goal of this team will be to investigate what we believe to be the possible technical solutions for initiatives and advise the team on what they believe would be the appropriate solution. To this end, we will reach out when we start looking for ideas as you may have ideas that we did not think about. The first investigation, led by Will Woods, Mark O'Brien and I, will be around datanommer and datagrepper. datanommer is an application listening to fedmsg and filling a (postgresql) database with all the messages passing on the bus. datagrepper is a web application exposing these messages and offering a way to filter or search them. available at: https://apps.fedoraproject.org/datagrepper/ Currently our ideas are: - for datanommer: - port it to fedora-messaging - adjust it to whichever solution we chose to replace datagrepper - for datagrepper: - keep it as is - Replace by - postgres https://postgrest.org/ - prest https://github.com/prest/prest - kinto https://docs.kinto-storage.org/en/stable/ - Swagger/OpenAPI https://swagger.io/ - Add support for Graphql - for the postgresql server - Split messages per year in different table - Unite them using a postgresql view - Kick out the old messages per year - Keep the current year + n-1 in the current DB - Kick the other to another DB? - Kick the other to a tarball somewhere? - Output the database daily dump to file / year - TimescaleDB a postgresql plugin for time-series data - https://alibaba-cloud.medium.com/postgresql-time-series-database-plug-in-ti… - https://dev.t-matix.com/blog/postgresql-as-a-time-series-database/ - https://docs.timescale.com/latest/introduction - Make the msg field in the message table be a JSON field Would you have any other ideas of things we could look at? Looking forward for your input, Thanks, Pierre, Will and Mark * Our notes and documentation are hosted at: https://fedora-arc.readthedocs.io/en/latest/index.html

3 6

CPE Weekly Report: 2021-02-26
by Aoife Moloney 26 Feb '21

26 Feb '21

1 0

Meeting Agenda Item: Introduction Marcelo Piedra
by Marcelo Piedra 24 Feb '21

24 Feb '21

Hello, Here I complete the required information. - IRC handle: I believe by this you meant the username, else please correct me. *stonefeller* or *lodbrok* are my usernames. - Marcelo Piedra - Time Zone: EET +2 / Finland - I know Python and have tried frameworks like Flask and fastAPI, also have basic experience using postgres. Have worked for +3 years as a product manager in different startups and also in databases as a service company. Additionally, I have learned some docker. Moreover, I have used vue.js and of course HTML and CSS. At the moment I am attending a course about cybersecurity, started a month ago so at the moment learning about networking. - I am looking to get started in development, as I have learned to code by myself and have used different frameworks, deployed to the cloud and used github, I need to get the real experience and feel of it. so looking to start with easy tasks to get started and learn to get into more advanced stuff at a later stage. If the tasks are related to security then would be good, but if that requires more experience then at a later stage is fine. - I can contribute a minimum of 10 hours per week, but it's possible to add more hours depending on the week. All the best, Marcelo

1 1

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 24 Feb '21

24 Feb '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-02-25 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by Damian Tometzki 24 Feb '21

24 Feb '21

1 0

FBR: To add i3 spin pungi config to F34
by Mohan Boddu 23 Feb '21

23 Feb '21

Hi, i3 spin is a new deliverable for F34 and this PR [1] adds the needed config for generating the i3 live iso as part of nightly compose. Please give your +1's. [1] https://pagure.io/pungi-fedora/pull-request/1012 Thanks, Mohan Boddu.

2 1

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 17 Feb '21

17 Feb '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-02-18 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

server updates
by Kevin Fenzi 16 Feb '21

16 Feb '21

Greetings all. I thought I would let everyone know that I have applied updates all around on our servers, excepting a few: copr* retrace* os* notifs* I have also rebooted stg hosts. I am going to be doing some more (non outage causing) reboots this week as well. Please keep an eye out for any odd behavior or problems that might have been caused by updates. After Beta is out, we can look at doing a normal update/reboot cycle. I just wanted to make sure we are mostly updated going into freeze. kevin

2 2

CPE Feedback Survey
by Ant Carroll 16 Feb '21

16 Feb '21

Hey folks, Apologies for the delay in getting this out to you after the start of the year. Hopefully you've noticed the changes to communication since the results of the last survey we did in August. However, we know this is ever changing, people join or become inactive and so want to ensure we continue with making improvements that benefit us all. I'm here asking for your help with this again [image: 🙂] Here is a link to a very short survey we've put together to learn how your experiences have been with the CPE team since October 2020. If you could take the time (5mins max) to complete it for us it would be hugely valuable as we work on this continuous improvement - https://fedoraproject.limequery.com/4?lang=en The survey will remain open until Feb 17th (23:59 UTC). Cheers, Ant -- Ant Carroll Associate Manager, Software Engineering Red Hat Waterford <http://www.redhat.com> Communications House Cork Road, Waterford City ancarrol(a)redhat.com M: +353876213163 IM: ancarrol @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://www.redhat.com/>

2 1

CPE Weekly: 2021-02-14
by Aoife Moloney 15 Feb '21

15 Feb '21

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 10 Feb '21

10 Feb '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-02-11 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

Re-importing accounts in staging
by Aurelien Bompard 08 Feb '21

08 Feb '21

Hey folks! The AAA team would like to test a re-import of the accounts in staging. We have learnt of a way to speed up the import significantly (20 times) and we'd like to test it. For that we'll need to remove all existing accounts and start from scratch. It means that if you're currently testing your application in staging, your account will disappear for something between hours to a couple days. We're going to start the process in 4-5 hours. Please shout if you're in the middle of something and you'd prefer us to wait for tomorrow. Thanks! Aurélien

1 0

CPE Report: 2020-02-05
by Aoife Moloney 05 Feb '21

05 Feb '21

Hi Everyone, If you would like to see this report and toggle to the section you are most interested in, I would suggest visiting this link https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view and use the header bar on your left to skip to where you want to go! ## Initiative FYI Links Initiatives repo here: https://pagure.io/cpe/initiatives-proposal 2021 Quarterly Planning timetable here: https://docs.fedoraproject.org/en-US/cpe/time_tables/ so you know when I need it in by to review it. Details on initiative requesting/how to work with us on new projects here: https://docs.fedoraproject.org/en-US/cpe/initiatives/ ### Misc #### Conferences! * CentOS Dojo @ FOSDEM is on right now! Links to talks from Thursday are on the CentOS youtube channel and Rich is playing a blinder getting all the content uploaded in record time https://www.youtube.com/TheCentOSProject * NOTE: 'playing a blinder' means doing an excellent job for anyone unfamiliar with the term :) * Fedora has a booth as well @ FOSDEM this weekend! Make sure you stop by and say hi to all those great Fedorans who will be manning it this weekend https://chat.fosdem.org/#/room/#fedora-stand:fosdem.org ## Project Updates *The below updates are pulled directly from our CPE team call we have every week.* ### Fedora * Infra team are assisting with the testing of ipa/noggin for otp/other cases in stg * Their also doing a cleanup of a bunch of broken links on koji volume * Mass rebuild of rpms is done, modules are underway * FTBFS for the mass rebuild are filled ### CPE ARC TEAM (Community Platform Engineering Advanced Reconnaissance Team....Team) We have a new sub team in our team, led by Pingou, who are running advance investigations on some of the tech debt and bigger initiatives that the CPE team have in our backlog and they have been tackling Datanomer/Datagrepper tech debt first. The team have been partitioning the ‘messages’ table of datagrepper's DB, & hope to be able to test this setup next week * prod like in openshift https://datagrepper-monitor-dashboard.app.os.fedoraproject.org * prod like with a default delta of 3 days http://datagrepper.arc.fedorainfracloud.org/datagrepper/ * partitioned table + default delta of 3 days http://datagrepper-test.arc.fedorainfracloud.org/datagrepper/ * using the timescale postgresql plugin [not implemented yet] http://datagrepper-timescale.arc.fedorainfracloud.org ### Noggin/AAA * We faced some issues with IPA limits and tuning, and 2FA & still trying to figure out the best way to enforce 2FA with sudo. * We are getting closer to migrating from stg to prod and once the Fedora migration is complete, the CentOS accounts will be then imported. * NOTE: If you have an account in both CentOS & Fedora and have different email addresses associated with each, please update your preferred email address in your profile and look out for an email next week on your options. * The work tracker for this project can be found here https://github.com/orgs/fedora-infra/projects/6 ### Fedora Messaging Schemas * Elections pr reviewed https://pagure.io/elections/pull-request/90 * Next is Greenwave & waiverdb * Board the issues are tracked on are here https://github.com/orgs/fedora-infra/projects/7 ## CentOS Updates ### CentOS * CI team members are migrating Fedora-Infra and Fedora-apps namespace whcih is one of the last few before we shut down legacy cluster * There is also an investigation spike on Zabbix upgrade to current LTS version which will then be rolled-out on the CentOS Infra once complete ### CentOS Stream * Python39 built and ready to compose * Dist-git repos are regularly up to date * Repos are populated in the CentOS Stream GitLab instance and will be publically viewable in the coming weeks * Very detailed talks on CentOS Stream given by Brian Stinson & Brian 'Bex' Exelbierd are watchable now on the CentOS YouTube channel - check them out! ## Team Info ### Background: The Community Platform Engineering group, or CPE for short, is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. See our wiki page here for more information:https://docs.fedoraproject.org/en-US/cpe/ As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great weekend! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford

1 0

Planned Outage - mbs-backend - 2021-02-03 10:30 UTC
by Mark O'Brien 04 Feb '21

04 Feb '21

There will be an outage starting at 2021-02-03 10:30 UTC, which will last approximately 30 mins. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: date -d '2021-02-03 10:30UTC' Reason for outage: mbs-backend01.iad2.fedoraproject.org keeps reaching disk full so the disk needs to be grown. This involves a brief shutdown of the instance to enlarge the logical volume Affected Services: MBS Ticket link: https://pagure.io/fedora-infrastructure/issue/9619https://pagure.io/fedora-… Please join #fedora-admin or #fedora-noc on irc.freenode.net or add comments to the ticket for this outage above.

1 1

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 03 Feb '21

03 Feb '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-02-04 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

Re: Fedora packages site down
by Anatoli Babenia 02 Feb '21

02 Feb '21

Hi. I've deployed the static website to GitHub Pages at https://abitrolly.github.io/fedora-packages-static/ The search doesn't work, and there is no index to browse, so it is not very useful. Looks like the search is supposed to be Solr, which is not static.

2 3

Anitya (release-monitoring.org) 1.0.0 available
by Michal Konecny 01 Feb '21

01 Feb '21

Hi everyone, today I deployed a new version of Anitya on production [0]. I decided that Anitya is mature enough to have version 1.0.0. So here it is. And what this versions brings? Plenty of changes, here is the list of the most interesting ones: * Add preview mode Now you can try your changes before submitting them, on the edit and add project page is a new button "Test check" which will take the fields from the form and do a check for releases above them. Nothing is changed in the database during test check. * Flag pre-release versions Yes, you are reading it right. Anitya is now flagging versions that are considered unstable, it uses the version scheme recognition and above that you can add your own filter when editing project. * Message schema 2.0.0 The Anitya message schema now contains a new topic "anitya.project.version.update.v2". This topic will send message that has "upstream_versions" field which contains all the newly found versions, not only the latest one. And it also contains a "stable_versions" field, so you can look if some of the newly versions is stable or not. With this version "anitya.project.version.update" is now deprecated! * Add version filter for project Anitya now allows user to add their own version filter, if you see any bogus version, you can just edit project and add the string to filter (This will not delete any version that was already retrieved, but you can flag a project and ask admin to do it for you and it will never be retrieved again). * Project archiving Anitya 1.0.0 allows admins to archive projects if it seems reasonable (project dead upstream) for the sake of history. Archived projects can't be edited and are not checked for new versions, but still could be found in Anitya. * Projects menu is rewritten The projects menu now contains items that are more sensible to current state of Anitya, you can see projects that were successfully updated (sorted by the time of update from newest), failed to update (sorted by the number of failed attempts from highest number), never updated (incorrectly set up projects, where update was never successful, sorted by the date of creation from oldest) and archived projects. * Updated documentation The documentation was fully rewritten to reflect the current state of Anitya. User guide was added containing use cases that could be done by user. User admin guide was added for users in Anitya with Admin rights. And the Admin guide and Contribution guide was verified that these steps are working with current version of Anitya. If you want to see whole list of changes, see Anitya 1.0.0 release on GitHub [1]. I hope this release will bring joy to your life and solve at least some of the pain points people had with Anitya. Michal Mage from release-monitoring.org P.S.: If you want to try something in Anitya without the fear of breaking anything, you can try it on staging instance [2]. [0] - https://release-monitoring.org/ [1] - https://github.com/fedora-infra/anitya/releases/tag/1.0.0 [2] - https://stg.release-monitoring.org/

3 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure February 2021