About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
11 months, 1 week
epel7-infra and epel7 conflicts wrt pagure updates (or, can we have
an epel7-infra DistTag?)
by Neal Gompa
Hey all,
So as you all know, Pagure 5.0 was released[1]. With 5.0, I've taken
over primary maintenance of the package outside of epel7-infra, which
means that Fedora and EPEL7 packages are generally maintained by me
(with packaging changes generally synced from upstream), since
Pierre-Yves doesn't have time to maintain it. We've already received
the approval to re-sync pagure's EPEL7 package with Fedora to update
to 5.0[2].
However, this presents a bit of a problem. Pierre-Yves updates the
pagure package for epel7-infra before making the general release that
I can consume and push to Fedora and EPEL7. This wouldn't be an issue,
except that the DistTag for epel7-infra is the same as epel7 (.el7).
This means I'm locked out of that NEVR for EPEL7 updates, due to the
nature of Koji considering NEVR of builds globally unique. This is
frustrating because it means that I can't maintain the common spec
file for both Fedora and EPEL7 (which is in turn primarily maintained
upstream in the pagure git repo).
Of course, I can just bump the Release to be above the epel7-infra
build, but that means I'm playing NEVR games, and I don't want to do
that. And if there needs to be distinct changes for epel7-infra, this
is going to be an unpleasant fight.
I'd like to propose that we change epel7-infra so that the DistTag
defined in %dist is ".el7.infra" instead of ".el7". This will prevent
the conflict and have the advantage that the infra tag is higher by
default.
For this current update round, I'm going to be putting a higher
Release value because I need to anyway, as there's a bug in the way
pagure is done that breaks packaging in Fedora and openSUSE[3]. But in
the future, I'd like to have the freedom to use the correct NEVR
everywhere.
What do you all think?
[1]: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedora...
[2]: https://meetbot.fedoraproject.org/teams/epel/epel.2018-09-26-17.01.html
[3]: https://pagure.io/pagure/issue/3706
--
真実はいつも一つ！/ Always, there's only one truth!
4 years, 11 months
The trouble with torrents
by Kevin Fenzi
So, Fedora 29 Beta is out the door and mostly that went very smoothly as
always. With one exception: torrents.
Right now we are (were) using the last opensource version of
'bittorrent' python client to seed our torrents. However, I noticed
yesterday that it is no longer working. It's not just one bug, but a
number of them judging by the different tracebacks it's spewing.
As a stopgap to get Beta happy I installed and ran rtorrent on torrent01
in a tmux session to seed our beta torrents, but thats a curses app
thats not automateable.
So, we really need to figure a way forward here.
1. Get permission from Council/FESCo to drop torrents (To make that case
we need to show some graphs that not many people are using torrents
anymore).
2. Move our torrents to amazon cloudfront. This should be doable, but
means we are using a non free service so it would make me sad.
3. Try and find a open source seeding app that is manageable from
ansible/config files. (Good luck, they are all old, or gui only, etc).
4. Just use rtorrent in tmux to seed everything and have to manually
start/stop it when you add/remove any.
5. Some other clever idea someone has.
We can discuss this in meeting tomorrow, but I figured I would start a
thread here too.
kevin
4 years, 11 months
[PATCH] Add a RabbitMQ vhost for pubsub (fedmsg) and configure an HA policy
by Jeremy Cline
Since many of our applications will likely wish to use RabbitMQ for
private things like work queues, using separate vhosts is a good idea.
This sets up a single vhost to act as a fedmsg replacement (called
pubsub) and configures it for high availability.
The HA policy mirrors queues to all nodes in the cluster and enables
automatic synchronization with a batch size of 10K messages. Automatic
synchronization means that when a new node joins the cluster, all
messages from existing nodes are pushed to the new node, blocking queue
activity until mirroring is complete.
Signed-off-by: Jeremy Cline <jcline(a)redhat.com>
---
roles/rabbitmq_cluster/files/enabled_plugins | 1 -
roles/rabbitmq_cluster/tasks/main.yml | 38 +++++++++++++++-----
2 files changed, 29 insertions(+), 10 deletions(-)
delete mode 100644 roles/rabbitmq_cluster/files/enabled_plugins
diff --git a/roles/rabbitmq_cluster/files/enabled_plugins b/roles/rabbitmq_cluster/files/enabled_plugins
deleted file mode 100644
index 6a9f28b93..000000000
--- a/roles/rabbitmq_cluster/files/enabled_plugins
+++ /dev/null
@@ -1 +0,0 @@
-[rabbitmq_management,rabbitmq_auth_mechanism_ssl].
diff --git a/roles/rabbitmq_cluster/tasks/main.yml b/roles/rabbitmq_cluster/tasks/main.yml
index 468cbc5ac..1781eb2c4 100644
--- a/roles/rabbitmq_cluster/tasks/main.yml
+++ b/roles/rabbitmq_cluster/tasks/main.yml
@@ -52,15 +52,6 @@
- rabbitmq_cluster
- config
-- name: enable plugins
- copy: src=enabled_plugins dest=/etc/rabbitmq/enabled_plugins owner=root group=root mode=0644
- with_items:
- - rabbitmq.config
- - enabled_plugins
- tags:
- - rabbitmq_cluster
- - config
-
- name: deploy configuration
template: src={{item}} dest=/etc/rabbitmq/{{item}} owner=root group=root mode=0644
with_items:
@@ -96,6 +87,13 @@
- rabbitmq_cluster
- config
+- name: Enable the HTTP management console and SSL authentication plugins
+ rabbitmq_plugin:
+ names: rabbitmq_management,rabbitmq_auth_mechanism_ssl
+ tags:
+ - rabbitmq_cluster
+ - config
+
- name: start rabbitmq
service: name=rabbitmq-server state=started enabled=yes
tags:
@@ -106,3 +104,25 @@
tags:
- rabbitmq_cluster
- config
+
+- name: Configure the pubsub virtual host
+ rabbitmq_vhost:
+ name: /pubsub
+ state: present
+ tags:
+ - rabbitmq_cluster
+ - config
+
+- name: Configure the HA policy for queues on the pubsub virtual
+ rabbitmq_policy:
+ name: HA
+ apply_to: queues
+ pattern: .*
+ tags:
+ ha-mode: all
+ ha-sync-mode: automatic # Auto sync queues to new cluster members
+ ha-sync-batch-size: 10000 # Larger is faster, but must finish in 1 net_ticktime
+ vhost: /pubsub
+ tags:
+ - rabbitmq_cluster
+ - config
--
2.19.0
4 years, 11 months
Data on torrents usage
by Stephen John Smoogen
So I updated my stats from 2017 and made new graphs to show how usage
of torrents has progressed since 2012.
https://smooge.fedorapeople.org/stats-torrents/
Due to the messiness of the logs.. the data logged is in an hourly
file which gives a checksum of the image being shared without any
knowledge of what that image is. [This is because tracker is meant to
be open and allow anyone to track whatever they want.] The logs only
say things like the number of peers, and the number of clients asking
for data. It isn't clear if it is showing the amount of actual
downloads completed or not. I decided to say it was.. but if that is
wrong, oops.
In general, I believe we should not stop producing torrents that
people can use. Running a tracker and dedicated seeder for that
tracker may be less useful for us because of the amount of work it
takes to keep it going. The problems are that development on headless
opensource trackers and seeders ended in 5-7 years ago. Someone in
release engineering has to patch something each release to keep it
going. In looking at various distributions, we seem to be the only one
still running a tracker. The others just produce torrent files and
seem to let the users find peers where-ever they can.
--
Stephen J Smoogen.
4 years, 12 months
MirrorManager changes - report_mirror
by Adrian Reber
During the last months most mirror problems have been related to
report_mirror reporting that mirrors are up to date and the crawler then
marking the mirror as not up to date. Which resulted in mirrors being
added and removed from the mirrorlist every few hours.
As the crawler normally sees what the user sees, means that
report_mirror often reports the wrong mirror status.
I would like to ignore report_mirror results for non-private mirrors in
the future. For private mirrors report_mirror is the only way to know
which content it has, for public mirrors it is not really necessary as
the mirrors are crawled anyway and report_mirror does not contain enough
information anyway right now.
Any objections to ignore report_mirror reports for public mirrors
after the F29 release.
Adrian
4 years, 12 months
[Release] pagure 5.0
by Pierre-Yves Chibon
Good Morning Everyone,
On Monday, I released a new version of pagure: 5.0.
Here is its changelog:
5.0 (2018-09-24)
----------------
.. warning:: This release contains backward incompatible changes and fixes a CVE
- Pagure supports now python2 and python3 simultaneously (Thanks to Aurélien
Bompard and Neal Gompa for the testing)
- New UI deployed (thanks to Ryan Lerch)
- New dashboard page as index page when authenticated (Ryan Lerch)
- API listing items (projects, issues, pull-requests are now paginated (
Karsten Hopp)
.. warning:: Backward incompatible
- Enable private projects by default (Neal Gompa)
.. warning:: Backward incompatible
- Change the default and sample configuration to point to localhost-friendly
resources (Neal Gompa)
.. warning:: Backward incompatible
- Disable sending FedMsg notifications by default (Neal Gompa)
.. warning:: Backward incompatible
- Switch default authentication to `local` (Neal Gompa)
.. warning:: Backward incompatible
- Drop the dependency on python-pygments
- Drop the dependency on flask-multistatic
- Drop the dependency on python-trollius (in favor of python-trololio) (Neal
Gompa)
- Bump pygit2 requirement to 0.24.0 minimum
.. warning:: Backward incompatible
- Add support to re-open a pull-request (Karsten Hopp)
- Fix editing a file into a fork containing a namespace
- Allow creating a new API token based on an expired one
- New API endpoint to submit a pull-request
- Add support for making the issue tracker read-only
- Add a new API endpoint allowing to update watch status on a project
- Paginate the project lists on the front pages
- Let the reply button append instead of replacing
- Add a way to list all API tokens and don't restrict the info command (
in pagure-admin)
- Expand pagure-admin to allow using it to block an user
- Expand pagure-admin to allow adding new groups using it
- Allow viewing commits from a git tag
- Support viewing commits from a specific commit hash
- Add a hook that disables creating new branches by git push (Slavek Kabrda)
- Make API endpoint for creating new git branch have its own ACL
- Support sorting PR's by recent activity (ymdatta)
- Fix installing the API key reminder cron with systemd integration
- Add reactions to comments (Lubomír Sedlář)
- New API endpoint allowing to retrieve pull-requests based on their UUID
(Slavek Kabrda)
- Add an option to restrict emails sent to certain domains (Karsten Hopp)
- Integration with repospanner (Patrick Uiterwijk)
- Rework how git hooks work to rely on a single file rather than moving files
around (Patrick Uiterwijk)
- Add themes for pagure.io, src.fedoraproject.org (Ryan Lerch)
- Add themes for OpenSUSE (hellcp)
- Ensure remote PR are opened from a remote URL (CVE-2018-1002158 - reported by
Patrick Uiterwijk)
This release is available:
- In Fedora
- In Mageia
- In OpenSUSE
- soon in EPEL
This release has been deployed in staging: stg.pagure.io and
src.stg.fedoraproject.org and in production in pagure.io yesterday.
We ran into some interesting issues in production that we didn't see in staging,
leading to some bug fixes and a version 5.0.1 to be released soon.
I take this opportunity to advertise that pagure now has two mailing lists:
- pagure-devel to discuss about the development of pagure:
https://lists.pagure.io/admin/lists/pagure-devel.lists.pagure.io/
- pagure-announce to follow announcement about new releases of pagure
https://lists.pagure.io/admin/lists/pagure-announce.lists.pagure.io/
feel free to join them and keep in touch with pagure :)
Future pagure release announcement will be sent to:
pagure-announce(a)lists.pagure.io
Happy hacking!
Pierre
4 years, 12 months
Meeting Agenda Item: Introduction Your Name
by David Shier
Hello all,
My name is David Shier and I am coming back to fedora admin after a
year or two away. My iRC nick is odin2016. Up until about a month and a
half ago, I worked for Jackson in the US as a number of things, but
most relevant to this is my last two positions, as an admin on their
UNIX team, supported our RHEL (5,6,7) and AIX estates, and working in
on the more or less server operations team as a senior associate with
root access to the production environment. I am familiar with bash,
python, and perl, as well as Ansible, which I used at work and at home.
I'd like to improve my Ansible and python skills, as well as just
keeping my self sharp while I search for a job after my recent move to
Germany. As would be expected I am a linux user at home, having used
redhat/centos, Debian/ubuntu, and Archlinux, as well as a number of
forks of each, Fedora included, but I am also a distro hopper and am
currently messing around with elementary on my laptop, and once the
rest of my stuff gets here... whenever that is.... i'll be reimaging
the raspberry pi and tossing awx on there. At my last job I worked a
good bit with RHDS, and passing data into it from Active Directory, as
I have experience on both sides of that fence as well. I'm sure there
things here I could say, and probably should, but i've also blocked out
large bits of the last year or so, so there is that.
There are a number of tickets that I would *like * to work on, but
before i get to deep into that I need to go and read up on how thiings
int eh infra have changed since I last took a look, clone the ansible
repo and read though there, and so on. I also think I saw something
about updating somethings to python 3, or making sure they will work. I
may take a look at those as well. I thought I saw something about the
bots as well, but now I don't so may be I am crazy. I have worked with
them before, so if I see it I may just grab it.
I should be at the meeting Thursday (I have a thing at 17:00 CEST),
unless I did the math on time offset really wrong or something, and
will get myself added to the goby doc in the next little bit here.
If someone could pop me back into fi-apprentice, that would be great..
fas is odin2016 as well.
Thanks,
Dave Shier
4 years, 12 months
FBR: upgrading pagure to 5.0
by Pierre-Yves Chibon
Good Morning Everyone,
I just cut a new release of pagure: 5.0, the formal announce is pending
polishing things with mailman so we can include in the announce the new mailing
lists to discuss/follow pagure.
Pending this, I'd like to request a Freeze Break Request for upgrading pagure to
5.0.
It's not critical for the release of the beta tomorrow, and it saves one day (vs
waiting for Wed) for the upgrade which considering some deadlines intern to the
infra team may be good.
What do you think?
Thanks,
Pierre
4 years, 12 months
