About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
7 months, 2 weeks
SQLAlchemy integration in Flask
by Aurelien Bompard
Hey folks!
I'd like to open a can of worms: SQLAlchemy integration in Flask. It's a
long read but I hope you'll like it. I try not to be on the ranty side.
First, some context: we have been plagued by tech debt for a very long
time, maybe more than other development projects because web tech has
evolved very quickly in the past 15 years (after the big flatline of "I
need to be IE6-compatible", but I digress), and also because we're a small
team maintaining a ton of apps.
In the past, we've had to switch web frameworks a couple of times.
Turbogears 1, then Turbogears 2, then Flask, but we also have apps in
CherryPy (to this day!) and Pyramid (and somewhat Django although we're not
maintaining them). As a result we've grown very suspicious of framework
integration because that means more work when we (inevitably) have to
migrate to a different framework. I think it's partly why we chose Flask at
the time: it's a minimalist framework, therefore there will be less
integration bits to migrate away from. I don't agree entirely with this
point of view but I think the standardization has done us a lot of good.
We've also standardized on SQLAlchemy for our DB, which is great. But we
haven't standardized on how to integrate the two. There's an integration
library called Flask-SQLAlchemy, but our apps don't use it (to my
knowledge). I think that the reason is that Flask-SQLAlchmemy makes your
models unusable without Flask, and that triggers our "what if the framework
goes away" PTSD. Here's an example of what models look like with
Flask-SQLAlchemy:
from flask import Flaskfrom flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True, nullable=False
As you can see, the User class inherits from db.Model, something that the
Flask-SQLAlchemy extension provides. Same for the Column and the DB types.
I do understand why it would make us flinch to "tie" our models to the web
framework, even indirectly. (that said, if we want to migrate those models
from Flask-SQLAlchemy to native SQLAlchemy, it would be so trivial that
plain sed can handle it)
So we've tried to do the SQLAlchemy integration in Flask ourselves. This
has two downsides:
- it's not as easy as it looks, especially when migrations and Alembic come
into play
- we end up with many slightly different integrations, written by different
people or even the same person at different points in time.
Ironically, our attempt at avoiding tech debt has caused us more tech debt.
Flask-SQLAlchemy has however some pretty strong points:
- it's widely used and of good quality
- it's maintained by the same people who maintain Flask (although the same
cannot be said of Flask-Migrate, the Alembic integration)
- it gives us some shortcuts very common in web apps, like get_or_404 or
pagination support, that we also had to write ourselves with differing
implementations.
But if we don't want to go this route, I've extracted a lightweight way of
integrating SQLAlchemy in Flask that leaves the models usable without
Flask. I think it's a reasonable concern to separate entirely the models
from the web app framework, so I'm not advocating for Flask-SQLAlchemy
here. I'm advocating against each app having its own integration.
On top of that, while I don't see us moving away from Flask anytime soon, I
do see a reason why we would want to look at other web frameworks in the
near future, and that reason is async support. Flask 2.x has grown support
for async views, but it's not as good
<https://flask.palletsprojects.com/en/2.0.x/async-await/#when-to-use-quart...>
as other async-native frameworks such as FastAPI or Starlette, and
extensions are unlikely to support it. That said, using async with
SQLAlchemy requires doing away with quite a few convenience features of the
ORM, such as implicit queries on model attributes, so moving to an async
framework would require us to rewrite all the database access layer anyway.
Maybe this is not a good example after all.
Anyway, this long email is about finding a common ground for SQLAlchemy
integration in Flask, while taking into account our difficult experiences
with webframewoks in the past, but not being locked in them. Is there
something that I misrepresented here? Do you have opinions? Preferences?
Thanks!
Aurélien
1 year, 5 months
CPE Weekly Update – Week of December 13th – 17th
by Vipul Siddharth
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering)
Team. If you have any questions or feedback, please respond to this
report or contact us on #redhat-cpe channel on libera.chat
(https://libera.chat/).
If you wish to read this in form of a discourse post (or on our
commblog), check the post on discussion.fp.o:
https://discussion.fedoraproject.org/t/cpe-weekly-update-week-of-december...
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release
(mirrors, mass branching, new namespaces etc.). The ARC (which is a
subset of the team) investigates possible initiatives that CPE might
take on.
Update
------
### Fedora Infra
* Log4j CVE affects our Openshift 3, mitigated via
[fix](https://access.redhat.com/solutions/6578441)
* Deployed an Openshift Dedicated cluster on AWS still deciding
* Lots of spam cleared off mailing lists
### CentOS Infra including CentOS CI
* Various activities/preparation for CentOS Linux 8 EOL process
* Proposal for SIGs to [switch to
RHEL8](https://lists.centos.org/pipermail/centos-devel/2021-December/0987...
* Preparing needed steps to clear content and remove it from mirrorlist
* WIP for Meta group for SIGs, allowing to build+push their -release
pkgs (not having to depend on CentOS Stream team for this)
* Investigating [openshift template
issue](https://pagure.io/centos-infra/issue/555) for newer deployed
projects and jenkins
* Log4j CVE impact analysis (impacting OCP 4 if Elasticsearch is used,
which we are not using)
* Stream MVBE infra : renewal process for internal TLS cert
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.
Updates
-------
* Dealing with a slowdown in rpm signing
* Continuing the development Content Resolver enhancements
* Various cleanup and docs work at various places
## CentOS Duffy CI
Goal of this Initiative
-----------------------
Duffy is a system within CentOS CI Infra which allows tenants to provision and
access bare metal resources of multiple architectures for the purposes of
CI testing.
We need to add the ability to checkout VMs in CentOS CI in Duffy. We have
OpenNebula hypervisor available, and have started developing playbooks which
can be used to create VMs using the OpenNebula API, but due to the current state
of how Duffy is deployed, we are blocked with new dev work to add the
VM checkout functionality.
Updates
-------
* Foundation for background tasks in place
* Started work on authentication and the session workflow
* CLI polish
## OSCI – Distrobaker monitoring
Goal of this Initiative
-----------------------
This initiative is to improve the Distrobaker monitoring to monitor
side-tags and module builds. Distrobaker is a service which rebuilds
the CentOS 9 Stream Koji builds for RHEL 9 in Brew.
Updates
-------
* added a barebones implementation of gathering side tag metrics
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special
Interest Group that creates, maintains, and manages a high quality set
of additional packages for Enterprise Linux, including, but not
limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific
Linux (SL), Oracle Linux (OL).
EPEL packages are usually based on their Fedora counterparts and will
never conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror
manager and more.
Updates
-------
* No updates - team on time offs
Kindest regards,
CPE Team
--
Vipul Siddharth
He/His/Him
Fedora | CentOS CI Infrastructure Team
1 year, 5 months
mailing lists, spam, and... burn it all down?
by Matthew Miller
Spurred by a big recent spam attack
https://pagure.io/fedora-infrastructure/issue/10417...
I think we should:
1. For all lists with no legitimate traffic in the past year, deactivate
immediately.
2. For any list with only light recent traffic, ask if they really need a
separate list or if posting on another list like devel (or on Fedora
Discussion! ... yes I have an agenda here) would actually be sufficient.
If there's no response, deactivate. Or, of course, if the list members
say "yeah, we don't need it".
3. For any list with _just_ automated traffic (zombie meeting invites,
bugzilla mail), ask if anyone is using it and if a list is the best way.
(It might be!) But some are probably sitting there really doing nothing
except sending automated messages into the void forever.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
1 year, 5 months
Fedora Messaging 3.0.0 released
by Aurelien Bompard
Hey folks!
I published version 3.0.0 of Fedora Messaging this morning. It is a major
release, and the backwards-incompatible changes are:
- Queues created by the CLI are now non-durable, auto-deleted and
exclusive, as server-named queues are.
- It is no longer necessary to declare a queue in the configuration file: a
server-named queue will be created. Configured bindings which do not
specify a queue name will be applied to the server-named queue. This is
most useful for our public broker where folks had to set a UUID-named queue
in the config file before starting the consumer.
- Drop support for Python 2
- Drop the Twisted classes that had been flagged as deprecated. Drop the
deprecated Message._body property. Refactor the consuming code into the
Consumer class.
It should be a painless upgrade. Here are the release notes if you also
want to check out the other changes (features, bugfixes, etc):
https://fedora-messaging.readthedocs.io/en/stable/changelog.html
I'll now go on to build the RPM.
Cheers!
Aurélien
1 year, 5 months
