About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
7 months, 3 weeks
Another Rust MirrorManager experiment
by Adrian Reber
Our MirrorManager setup exports the current state of all mirrors every
hour at :30 to a protobuf based file which is then used by the
mirrorlist servers to answer the requests from yum and dnf.
The Python script requires up to 10GB of memory and takes between 35 and
50 minutes. The script does a lot of SQL queries and also some really
big SQL queries joining up to 6 large MirrorManager tables.
I have rewritten this Python script in Rust and now it only needs around
1 minute instead of 35 to 50 minutes and only 600MB instead of 10GB.
I think the biggest difference is that I am almost not doing any joins
in my SQL request. I download all the tables once and then I do a lot of
loops over the downloaded tables and this seems to be massively faster.
As the mirrorlist-server in Rust has proven to be extremely stable over
the last months we have been using it I would also like to replace the
mirrorlist protbuf input generation with my new Rust based code.
I am planing to try out the new protobuf file in staging in the next
days and would then try to get my new protobuf generation program into
Fedora. Once it is packaged I would discuss here how and if we want to
deploy in Fedora's infrastructure.
Having the possibility to generate the mirrorlist input data in about a
minute would significantly reduce the load on the database server and
enable us to react much faster if broken protobuf data has been synced
to the mirrorlist servers on the proxies.
Adrian
2 years, 7 months
Fedora 33 Beta Freeze now in effect
by kevin
Greetings.
We are now in the infrastructure freeze leading up to the Fedora 33
Beta release. This is a pre release freeze.
We do this to ensure that our infrastructure is stable and ready to
release the Fedora 33 Beta when it's available.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
git clone
https://infrastructure.fedoraproject.org/infra/ansible.git
ansible/scripts/freezelist -i inventory
Any hosts listed as freezes is frozen until 2020-09-15 (or later if
release slips). Frozen hosts should have no changes made to them without
a sign-off on the change from at least 2 sysadmin-main or rel-eng
members, along with (in most cases) a patch of the exact change to be
made to this list.
Thanks,
Kevin
2 years, 8 months
What is our technical debt?
by Pierre-Yves Chibon
Good Morning Everyone,
Just like every team we have technical debt in our work.
I would like your help to try to define what it is for us.
So far, I've come up with the following:
- python3 support/migration
- fedora-messaging
- fedora-messaging schema
- documentation
- (unit-)tests
- OpenID Connect
What else would we want in there?
Looking forward to your thoughts,
Pierre
2 years, 8 months
IPAM/DCIM for Fedora Infrastructure
by Pierre-Yves Chibon
ignatenkobrain wanted to share this with us as something potentially interesting
to look at.
He opened a ticket but agreed that an email thread would be better suited to
discuss this, so here it is and here below is Igor's original comment:
----
Hi folks,
This is not really request to do anything but rather heads-up from work I've
been doing in GoodData for some time. We've been searching for something that
helps us to have list of devices, their IPs and generally some kind of IP
management in one place. So we started to use netbox (I think it is originating
from DigitalOcean).
I've finally managed to package it https://src.fedoraproject.org/rpms/netbox in
Fedora. It supports LDAP and I guess can be somehow paired with Kerberos. I did
not check if it is possible to pair it with OIDC, but since it is Django should
be not very hard.
So in case you were looking for something like this - feel free to try it out
and let me know if you have any issues.
----
Original ticket: https://pagure.io/fedora-infrastructure/issue/9279
Pierre
2 years, 9 months
The approach to extract all Fedora srpm?
by Jean-Baptiste Holcroft
Hello,
I would like to provide translation memories for translators and measure localization progress over Fedora versions (Linux wide).
I need to get upstream's content to get translation files such as po/gettext files.
To do that, I need to learn how to interact with Fedora packages to reach the SRPM.
The easiest way would probably be to run `dnf list --all` in a virtual machine or container for each Fedora release.
An alternative could be to use datagrepper using org.fedoraproject.prod.buildsys.build.state.change? Using this method would allow me to catch new changes, but how do I load previous events or do the first initialization?
Is there other ways to get all the SRPM of a Fedora release?
thanks a lot for your help,
I had a discussion on fedora-i18n [0], and Ben Cotton suggested me to ask my question here.
I tried to document what I want to do in this wiki page page [1]
[0] https://lists.fedoraproject.org/archives/list/i18n@lists.fedoraproject.or...
[1] https://fedoraproject.org/wiki/User:Jibecfed/LinuxLocalizationMeasurement
2 years, 9 months
Freeze Break Request (Fix 2factor on builders)
by Stephen John Smoogen
Currently the builders are trying to contact os-node boxes on port 8443 but
that is not where fas-all lives.
diff --git a/roles/base/templates/iptables/iptables.kojibuilder
b/roles/base/templates/iptables/iptables.kojibuilder
index a3819777c..805cf735f 100644
--- a/roles/base/templates/iptables/iptables.kojibuilder
+++ b/roles/base/templates/iptables/iptables.kojibuilder
@@ -78,10 +78,12 @@
-A OUTPUT -p tcp -m tcp -d 10.3.163.76 --dport 443 -j ACCEPT
-A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 80 -j ACCEPT
-A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 443 -j ACCEPT
-# for 2 facter auth
--A OUTPUT -p tcp -m tcp -d 10.3.163.69 --dport 8443 -j ACCEPT
--A OUTPUT -p tcp -m tcp -d 10.3.163.70 --dport 8443 -j ACCEPT
--A OUTPUT -p tcp -m tcp -d 10.3.163.71 --dport 8443 -j ACCEPT
+# for 2 facter auth (fas-all)
+-A OUTPUT -p tcp -m tcp -d 10.3.163.74 --dport 8443 -j ACCEPT
+-A OUTPUT -p tcp -m tcp -d 10.3.163.75 --dport 8443 -j ACCEPT
+-A OUTPUT -p tcp -m tcp -d 10.3.163.76 --dport 8443 -j ACCEPT
+-A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 8443 -j ACCEPT
+
#nfs to vtap-fedora-nfs01.storage.phx2.redhat.com - a little to wide-open
- but
# kinda necessary
--
Stephen J Smoogen.
2 years, 9 months
Meeting Agenda Item: Introduction Manu Hernandez
by Manu Hernandez
FAS: manher
IRC: miausX or manher
Hi there!
My name is Manu.
I have been the sysadmin/tech support for a small manufacturing company
(about 100 employees) for twelve years.
Our IT team is rather small (two developers and me) and our
infrastructure is very simple: just a few servers and networking gear.
Nothing fancy.
As in most companies in the area, ours relies on Microsoft products, so
the core services are Windows based. We use VMware vSphere to manage our
virtual infrastructure.
I love FLOSS. Not only the software (the product), but the culture. It's
amazing what a community can achieve. So many people giving away it's
free time, effort and expertise...
I try to learn about and use FLOSS solutions as much as I can (and time
permits). I have being a Linux user since 2003 (with some hiatus and
lots of distro-hopping), and I use Fedora on all my computers, both at
home and at work.
My current homelab runs CentOS as a KVM hypervisor, and I plan to run
all the workloads with Fedora CoreOS and podman.
Also, I try to automate as much as I can, being Ansible my tool of
choice. I use Kickstart to deploy to bare metal and virtual machines.
I don't have experience with programming languages (very basic scripting
skills at most) but I'd love to learn Python.
So, why do I want to join the Fedora infrastructure team?
For two reasons: I want to give back to this amazing community and I
want to learn as much as I can from the team.
I'm a Jack-of-all-trades and I know I don't have much to offer to this
team, but my curiosity, tenacity and willingness to learn.
If I'm not a good fit for this team, don't worry! Just let me know. I'll
try to help in another way to the project.
Pointers and notes to improve my skill set are more than welcome.
Nice to meet you all, and thanks for reading!
Manu
PS - English is not my native language. I can read it without problems
but I'm a bit slow with the writing and have difficulties with the
listening. I'll try to improve that too!
-- Email domain proudly hosted at https://migadu.com
2 years, 9 months
Meeting Agenda Item: Tom Collins
by Tom Collins
IRC/FAS: TomAPC
Hey folks,
This is a bit of a reactive mailing post, as I joined the weekly Fedora
infra meeting earlier today, so apologies for that :)
I'm Tom, a SysAdmin based in the UK looking to get involved in FOSS whilst
utilizing (and growing) my current skill set.
I have had experience with Docker/K8s, basic DevOps practices, Linux server
and client SysAdmin (mostly Debian/Ubuntu based, admittedly) and have a
history of tech writing and project/program management, so definitely
looking forward to the Infra Doc Hackfest happening next week!
I'm looking mostly to expand my knowledge outside of the in-house tooling
we use at work right now, and see how the infrastructure that is relied on
by millions of machines works!
Happy to take any further advice on how to start (I think Infrastructure
Apprentice is the first step? Let me know if I read the docs wrong!).
Look forward to working with you all.
Thanks,
Tom C
2 years, 9 months
