About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
1 year, 1 month
Introducing the ARC sub-team in CPE - and first research topic
by Pierre-Yves Chibon
Good Morning Everyone,
While planning work, the CPE team has realized that a number of our initiatives
actually start with a research phase to find the most appropriate technical
solution.
This leads to some issues with planning as without knowing the technical
solution we want to take, it's hard to evaluate the amount of work needed and
thus the time it'll take to do it.
In order to help with this, we're creating a small sub-team in CPE, called the
ARC team for Advance Reconaissance Crew*.
The goal of this team will be to investigate what we believe to be the possible
technical solutions for initiatives and advise the team on what they believe
would be the appropriate solution.
To this end, we will reach out when we start looking for ideas as you may have
ideas that we did not think about.
The first investigation, led by Will Woods, Mark O'Brien and I, will be around
datanommer and datagrepper.
datanommer is an application listening to fedmsg and filling a (postgresql)
database with all the messages passing on the bus.
datagrepper is a web application exposing these messages and offering a way to
filter or search them.
available at: https://apps.fedoraproject.org/datagrepper/
Currently our ideas are:
- for datanommer:
- port it to fedora-messaging
- adjust it to whichever solution we chose to replace datagrepper
- for datagrepper:
- keep it as is
- Replace by
- postgres https://postgrest.org/
- prest https://github.com/prest/prest
- kinto https://docs.kinto-storage.org/en/stable/
- Swagger/OpenAPI https://swagger.io/
- Add support for Graphql
- for the postgresql server
- Split messages per year in different table
- Unite them using a postgresql view
- Kick out the old messages per year
- Keep the current year + n-1 in the current DB
- Kick the other to another DB?
- Kick the other to a tarball somewhere?
- Output the database daily dump to file / year
- TimescaleDB a postgresql plugin for time-series data
- https://alibaba-cloud.medium.com/postgresql-time-series-database-plug-in-...
- https://dev.t-matix.com/blog/postgresql-as-a-time-series-database/
- https://docs.timescale.com/latest/introduction
- Make the msg field in the message table be a JSON field
Would you have any other ideas of things we could look at?
Looking forward for your input,
Thanks,
Pierre, Will and Mark
* Our notes and documentation are hosted at:
https://fedora-arc.readthedocs.io/en/latest/index.html
2 years, 9 months
Anitya (release-monitoring.org) 1.0.0 available
by Michal Konecny
Hi everyone,
today I deployed a new version of Anitya on production [0]. I decided
that Anitya is mature enough to have version 1.0.0. So here it is.
And what this versions brings? Plenty of changes, here is the list of
the most interesting ones:
* Add preview mode
Now you can try your changes before submitting them, on the edit and add
project page is a new button "Test check" which will take the fields
from the form and do a check for releases above them. Nothing is changed
in the database during test check.
* Flag pre-release versions
Yes, you are reading it right. Anitya is now flagging versions that are
considered unstable, it uses the version scheme recognition and above
that you can add your own filter when editing project.
* Message schema 2.0.0
The Anitya message schema now contains a new topic
"anitya.project.version.update.v2". This topic will send message that
has "upstream_versions" field which contains all the newly found
versions, not only the latest one. And it also contains a
"stable_versions" field, so you can look if some of the newly versions
is stable or not. With this version "anitya.project.version.update" is
now deprecated!
* Add version filter for project
Anitya now allows user to add their own version filter, if you see any
bogus version, you can just edit project and add the string to filter
(This will not delete any version that was already retrieved, but you
can flag a project and ask admin to do it for you and it will never be
retrieved again).
* Project archiving
Anitya 1.0.0 allows admins to archive projects if it seems reasonable
(project dead upstream) for the sake of history. Archived projects can't
be edited and are not checked for new versions, but still could be found
in Anitya.
* Projects menu is rewritten
The projects menu now contains items that are more sensible to current
state of Anitya, you can see projects that were successfully updated
(sorted by the time of update from newest), failed to update (sorted by
the number of failed attempts from highest number), never updated
(incorrectly set up projects, where update was never successful, sorted
by the date of creation from oldest) and archived projects.
* Updated documentation
The documentation was fully rewritten to reflect the current state of
Anitya. User guide was added containing use cases that could be done by
user. User admin guide was added for users in Anitya with Admin rights.
And the Admin guide and Contribution guide was verified that these steps
are working with current version of Anitya.
If you want to see whole list of changes, see Anitya 1.0.0 release on
GitHub [1].
I hope this release will bring joy to your life and solve at least some
of the pain points people had with Anitya.
Michal
Mage from release-monitoring.org
P.S.: If you want to try something in Anitya without the fear of
breaking anything, you can try it on staging instance [2].
[0] - https://release-monitoring.org/
[1] - https://github.com/fedora-infra/anitya/releases/tag/1.0.0
[2] - https://stg.release-monitoring.org/
2 years, 10 months
LISA CfP open
by Ben Cotton
Hi everyone,
The CfP for the LISA Conference is open through 23 February:
https://www.usenix.org/conference/lisa21/call-for-participation
It is a virtual event this year, held the first three days of June. I
was involved with this conference for several years in a past life and
it's a great venue for sharing sysadmin/DevOps knowledge.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
2 years, 10 months
Self introduction email
by Mauro Filanti
Hi guys!
I already sent an email to the fedora-join mailing list so here my self
introduction:
-
https://lists.fedoraproject.org/archives/list/fedora-join@lists.fedorapro...
Anyway I'm writing here in the infrastructure mailing list because it's one
of my interests and where I think I can start to help/contribute in
some way (I'll send it to the cloud one too, sorry if you receive it from
both the mailing lists).
My experience as Sysadmin it's new, just a few months old, but I've always
been interested in the admin side of the system so please if someone can
help me in order to get the next steps, would be great!
Thanks and stay safe!
Mauro
2 years, 10 months
Resources for a hosting an alternative Fedora buildroot
by Tom Stellard
Hi,
I am considering making a proposal for an alternative buildroot (similar
to ELN) in Fedora, and I want to try to understand what Fedora resources
might be available for this.
My requirements are:
+ A koji server and builders for doing 'shadow' builds of ~2000 Fedora
packages. 'shadow' builds means that every time a build is made in the
official Fedora koji then one would be done for this alternative buildroot.
+ Builders for all architectures supported by Fedora.
+ Developer access to the koji server using Fedora kerberos credentials.
+ Permissions to create and modify tags on the koji server for at least
3 people.
+ A DNF repo generated from these package builds that is updated once
per day.
+ Jenkins server that can host a CI job that listens for Fedora builds
and submits builds to this alternative buildroot.
My questions are:
Do we have enough machine resources and people time to implement
something like this, and if not, how much additional resources would be
required?
How and where to implement this, e.g. can we use the staging koji server
or something else?
Can we build something that is generic enough, so that other people who
want to make similar proposals would be able to easily set up their own
alternative buildroot?
Thanks,
Tom
2 years, 10 months
CPE Weekly: 2021-01-15
by Aoife Moloney
Hi Everyone,
New Year, same CPE weekly(ish) :)
If you would like to see this report and toggle to the section you are
most interested in, I would suggest visiting this link
https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view and use the header bar
on your left to skip to where you want to go!
## General Project Updates
We are kicking off Q1 this year with some familiar project faces,
namely Noggin, the replacement of the current FAS system and
continuing our development of CentOS Stream.
Most of our initiatives live here
https://pagure.io/cpe/initiatives-proposal and you can use the new
issue button to submit your own proposal.
Our updated initative timetable can be viewed here for 2021
https://docs.fedoraproject.org/en-US/cpe/time_tables/ so you know when
I need it in by to review it.
We also have updated our docs section on the initiative process we
follow as we cannot accept everything so please do check it out if you
want to understand our process more
https://docs.fedoraproject.org/en-US/cpe/initiatives/
### Misc
#### GitLab
Being very honest, I've found myself a little bit strapped for time to
give this project its due diligence over the last few months, but
please bear with us/me and expect a more concentrated effort on this
coming into Q2 (April, May, June) of this year. I apologise for the
time a resolution is taking and I really do appreciate all of your
patience.
## Project Updates
*The below updates are pulled directly from our CPE team call we have
every week.*
### Fedora
* OSBS is building for aarm64 & x86_64 in production since December!
* All of the projects under the fedora-infra and releng namespaces on
pagure have had their default branch migrated from “master” to “main”.
* F34 mass rebuild due to start next week
### Noggin/AAA
* New sprint started focusing on testing correct access has been given
per user/account
* Last remaining apps being configured & tested with fasjson API
* Work will be tracked here https://github.com/fedora-infra/aaa-tracker/issues/4
* Our open issues board can be found here
https://github.com/orgs/fedora-infra/projects/6
### Fedora Messaging Schemas
* We are working through supybot and greenwave applications currently
* There is a list of applications that require messaging schemas can
be found here https://hackmd.io/@nilsph/H1i8CAbkP/edit
* There is a readme which contains documentation on messaging schemas,
a cookie-cutter template to create the schema and a definition of Done
for writing a schemas
https://github.com/fedora-infra/fedora-messaging-schemas-issues
* The board they are working from can be viewed here
https://github.com/orgs/fedora-infra/projects/7
## CentOS Updates
### CentOS
* Community newsletter can be read here
https://blog.centos.org/2021/01/centos-community-newsletter-january-2020-...
### CentOS Stream
* Continuing to work on Stream 8 pushes and builds
* Investigating how to automate some module pushes
* Reviewing documentation that is available on Stream currently to
identify gaps and where needs improvement
## Team Info
### Background:
The Community Platform Engineering group, or CPE for short, is the Red
Hat team combining IT and release engineering from Fedora and CentOS.
Our goal is to keep core servers and services running and maintained,
build releases, and other strategic tasks that need more dedicated
time than volunteers can give.
See our wiki page here for more
information:https://docs.fedoraproject.org/en-US/cpe/
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great weekend!
Aoife
Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
2 years, 10 months
