About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
7 months, 2 weeks
Another Rust MirrorManager experiment
by Adrian Reber
Our MirrorManager setup exports the current state of all mirrors every
hour at :30 to a protobuf based file which is then used by the
mirrorlist servers to answer the requests from yum and dnf.
The Python script requires up to 10GB of memory and takes between 35 and
50 minutes. The script does a lot of SQL queries and also some really
big SQL queries joining up to 6 large MirrorManager tables.
I have rewritten this Python script in Rust and now it only needs around
1 minute instead of 35 to 50 minutes and only 600MB instead of 10GB.
I think the biggest difference is that I am almost not doing any joins
in my SQL request. I download all the tables once and then I do a lot of
loops over the downloaded tables and this seems to be massively faster.
As the mirrorlist-server in Rust has proven to be extremely stable over
the last months we have been using it I would also like to replace the
mirrorlist protbuf input generation with my new Rust based code.
I am planing to try out the new protobuf file in staging in the next
days and would then try to get my new protobuf generation program into
Fedora. Once it is packaged I would discuss here how and if we want to
deploy in Fedora's infrastructure.
Having the possibility to generate the mirrorlist input data in about a
minute would significantly reduce the load on the database server and
enable us to react much faster if broken protobuf data has been synced
to the mirrorlist servers on the proxies.
Adrian
2 years, 6 months
What is our technical debt?
by Pierre-Yves Chibon
Good Morning Everyone,
Just like every team we have technical debt in our work.
I would like your help to try to define what it is for us.
So far, I've come up with the following:
- python3 support/migration
- fedora-messaging
- fedora-messaging schema
- documentation
- (unit-)tests
- OpenID Connect
What else would we want in there?
Looking forward to your thoughts,
Pierre
2 years, 8 months
CPE Weekly: 2020-06-28
by Aoife Moloney
---
title: CPE Weekly status email
tags: CPE Weekly, email
---
# CPE Weekly: 2020-06-218
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.
See our wiki page here for more
information:https://docs.fedoraproject.org/en-US/cpe/
## General Project Updates
The CPE team have finished our Quarterly Planning for Q3, July -
September, and will begin work on the following projects starting from
Monday 5th July:
* Data Centre Move - Final Works
* CentOS Stream Phase 3
* Noggin Phase 3
* Packager Workflow Healthcare
* Fedora Messaging Schemas
Details of the above projects, and of projects currently in progress,
done and what projects are in our backlog, can be found on our taiga
board per project card:
https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null
We also have an updated initiative timetable for briefing in new
projects to our team & key dates
here:https://docs.fedoraproject.org/en-US/cpe/time_tables/
*Note: Initiatives are large pieces of work that require a team of
people and weeks/months to complete. Please continue to open tickets
in the normal way for bugs, issues, etc.
### CPE Product Owner Office Hours
#### #fedora-meeting-1
* Weekly on Thursdays @ 1300 UTC on #fedora-meeting-1
* Meeting log from 2020-06-25:
https://meetbot.fedoraproject.org/teams/cpe_project_owner_office_hours/cp...
* Next Meeting: 2020-07-02
#### #centos-meeting
* Every second Tuesday @ 1500 UTC on #centos-meeting
* Meeting log from 2020-06-23:
https://www.centos.org/minutes/2020/June/centos-meeting.2020-06-23-15.00....
* Next Meeting: 2020-07-07
## Fedora Updates
### Data Centre Move
* We are now officially operating under reduced Fedora services until
est 28th July to facilitate the final shipment of hardware to the new
data centre.
* Please View project card on taiga for builder - staging - remaining
app bringup dates here
https://tree.taiga.io/project/amoloney1-cpe-team-projects/us/42?kanban-st...
* A list of affected services is available here
https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view
* Details on what this move may mean for you can be found here
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
* If an application is not working correctly at all, please check this
list https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view before opening a
ticket to make sure its not listed as being moved. If it is being
moved, please wait a day or two, then try again.
* Similarly, please be patient when opening tickets for service issues
in general as we have now reached the critical point in this move and
all of our sys-admins and wider teams will be assisting in the
successful bringup of the reduced Fedora service and facilitation of
the final hardware shipment and move.
* Most recent update to devel-announce is here
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
### AAA Replacement
* The team are finishing some work on applications including
* Codebase changes to applications
* Application maintainers can use the API to look up group and
account information
* And addressing how to add a spam blocking feature
* Please feel free to check out the team kanban board for more
information on the features the team are working on and have already
completed here https://github.com/orgs/fedora-infra/projects/6
### Mbbox
* Project Dashboard here https://github.com/fedora-infra/mbbox/projects/1
* Tasks completed in the project currently
* MBBox handover to CentOS Stream
* Mbbox has been deployed to the staging environment
* Cleaning old code from the project was done
* Deployment guide - https://mbbox-operator.readthedocs.io/en/latest/
* Blog post published - check it out here
https://communityblog.fedoraproject.org/mbbox-module-building-in-a-box/
* Tasks expected to be complete by 3rd July
* MBox shared CRD - tests and polishing
### Gitforge
We are still discussing technical aspects of the project and these are
tracked here:
https://gitlab.com/gitlab-org/gitlab/-/issues/217350
And are working on a date in August/September for an AMA session with
GitLab that will be run through IRC for any questions the Fedora and
CentOS communities may have to ask direct.
We will keep you up to date with the developments as and when we have
information to share and thank you for your patience.
## CentOS Updates
### CentOS
* CentOS Linux 8.2.2004 has been released
https://lists.centos.org/pipermail/centos-announce/2020-June/035756.html
* CentOS 7.x AMI images to land soon
* Infra Nodes are also moving
* The CentOS CPE team members are also working on a new website design
with the community, so this should be pushed soon too.
### Centos Other
* Fedora CI and Noobaa tenants have been onboarded to CentOS CI infra
* The team are also looking at automation work within CI
### CentOS Stream
* RealTime for Stream is built!
* The team have also finished laying out repos and are doing final
tests with the installer
* This will be pushed to the mirrors very soon so keep an eye out!
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great week ahead!
Aoife
Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
2 years, 11 months
mirrorlist-server fix is in testing
by Adrian Reber
Currently there is a bug open against our mirrorlist servers that
certain locations have trouble, especially with the
fedora-cisco-openh264 repository, getting a list of up to date mirrors.
https://bugzilla.redhat.com/show_bug.cgi?id=1844087
This has been fixed upstream and is available from testing:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-f86d3b3196
As Fedora Infrastructure is, so far, the only user of this package
nobody besides us can/will test it.
Can we install it on one of our proxies and see if it still works and if
it actually fixes the problems which have been reported?
I cannot install anything on the proxies myself, so somebody would need
to install that package and restart the corresponding services and let
me know. There is no staging, but maybe there are proxies which are
currently not active which could be used for a test like this.
Adrian
2 years, 11 months
Demo of new Fedora Packages
by Miroslav Suchý
It took longer than expected, but I have something to show. Finally!
Live demo:
http://35.170.54.152/
Some tabs may not work now because the server are being migrated.
Search database is snapshot from May and may not contain all packages.
Main changes:
* Python3 !
* usage of PatternFly
* removed TG and Moksha, replaced by Flask.
* Tab "sources" has not been migrated and there is just link to src.f.o - no need to duplicate pagure.
The code is here:
https://github.com/xsuchy/fedora-packages-ng/
There is still some work to do. Mainly fedmsg->fedora-messaging, but rest should be just polishing.
Feedback is welcomed.
Credits go to Jun Aruga and Jakub Kadlcik who both participated on this.
--
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
2 years, 11 months
CPE Weekly: 2020-06-21
by Aoife Moloney
---
title: CPE Weekly status email
tags: CPE Weekly, email
---
# CPE Weekly: 2020-06-21
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.
See our wiki page here for more
information:https://docs.fedoraproject.org/en-US/cpe/
## General Project Updates
Please check out our updated initiative timetable for briefing in new
projects to our team
here:https://docs.fedoraproject.org/en-US/cpe/time_tables/
*Note: Initiatives are large pieces of work that require a team of
people and weeks/months to complete. Please continue to open tickets
in the normal way for bugs, issues, etc.
Dont forget to view our taiga board to see the projects we are
currently working on, what we have scoped and whats in our backlog
https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null
CPE Product Owner Office Hours
* Weekly on Thursdays @ 1300 UTC on #fedora-meeting-1
* Every second Tuesday @ 1500 UTC on #centos-meeting (next meeting 23rd June)
## Fedora Updates
### Data Centre Move
* We are now officially operating under reduced Fedora services until
est 28th July to facilitate the final shipment of hardware to the new
data centre.
* Most recent update to devel-announce is here
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
* A list of affected services is available here
https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view
* Details on what this move may mean for you can be found here
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
* If an application is not working correctly at all, please check this
list https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view before opening a
ticket to make sure its not listed as being moved. If it is being
moved, please wait a day or two, then try again.
* Similarly, please be patient when opening tickets for service issues
in general as we have now reached the critical point in this move and
all of our sys-admins and wider teams will be assisting in the
successful bringup of the reduced Fedora service and facilitation of
the final hardware shipment and move.
### AAA Replacement
* The team are working on the changing application code bases to use
the new solution
* The work required to facilitate multiple contributor agreements
under one account is now complete
* The team have also enabled group attributes during this sprint
* Please feel free to check out the team kanban board for more
information on the features the team are working on and have already
completed here https://github.com/orgs/fedora-infra/projects/6
### Mbbox
* Project Dashboard here https://github.com/fedora-infra/mbbox/projects/1
* Tasks completed in the project currently
* MBS Frontend CRD + documentation
* Staging environment - thanks to CentOS CI team
* MBox shared CRD - certificates and PVCs
* Tasks underway currently
* Staging environment - testing
* MBox shared CRD - tests and polishing
### Gitforge
With the data centre move taking most of the teams focus this week,
discussions with gitlab have been quiet. We are still discussing
technical aspects of the project and these are tracked here:
https://gitlab.com/gitlab-org/gitlab/-/issues/217350
We will keep you up to date with the developments as and when we have
information to share and thank you for your patience.
## CentOS Updates
### CentOS
* CentOS Linux 8.2.2004 Released!!!
https://lists.centos.org/pipermail/centos-announce/2020-June/035756.html
### Centos Other
* OCP4 staging cluster is up and running
* CentOS will have a booth at Open Source Summit starting June 29th -
https://events.linuxfoundation.org/open-source-summit-north-america/
### CentOS Stream
* The team are still working on building packages from the RHEL 8.3
nightly composes
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great week ahead!
Aoife
Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
2 years, 11 months
Meeting Agenda Item: Introduction Francois Andrieu
by Andrieu Francois
Hello everyone o/
I'm a system engineer for a little bit more than 10 years now, and would like to contribute to the Fedora Project in any way I can.
I've started learning about Ansible since last year and loving it so far.
This team seems a good place to learn more about it, and develop tons of playbooks :)
I've also some skills in Python which I look forward to improve by contributing here.
I've some RH certs in Linux, Ansible, Openshift and hold a RHCA in Infrastructure.
I'll probably just read some docs and explore some code to sink in for now, but I hope to be able to help you in any way some day.
See you on irc (handle : darknao)
Have a nice day!
2 years, 11 months
