... or even if you didn't like it, it still probably needs a lock.
/usr/local/bin/pagure-sync-bugzilla.py takes more than 24 hours to
run, and it doesn't have a lock wrapper on it. It needs one.
diff --git a/roles/distgit/pagure/tasks/main.yml b/roles/distgit/pagure/tasks/main.yml
index 8cdf0b3..9079f06 100644
@@ -228,7 +228,7 @@
- job: /usr/local/bin/pagure-sync-bugzilla.py
+ job: /usr/bin/lock-wrapper pagure-sync-bugzilla "/usr/local/bin/pagure-sync-bugzilla.py"
when: env != 'staging'
Anyone for it? Against it?
we are now in the infrastructure freeze leading up to the Fedora 27 Beta
release. This is a pre-release freeze.
We do this to ensure that our infrastructure is stable and ready to
release the Fedora 27 Beta when it's available.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
ansible/scripts/freezelist -i inventory
Any hosts listed as freezes is frozen until 2017-09-19 (or later if Beta
slips). Frozen hosts should have no changes made to them without a
sign-off on the change from at least 2 sysadmin-main or rel-eng members,
along with (in most cases) a patch of the exact change to be made to
I had a brief conversation with Pingou a few weeks ago during the
elections about deploying a new version of the Elections app with some
of the UI/UX rework that Ryan Lerch worked on last year.
The current codebase can be found here:
Is there a way the current version could be put into a staging instance?
Justin W. Flory
I'd like to propose that we build a newer version of pyOpenSSL for EL7.
The version provided by base RHEL is 0.13.1. We need at least 16.1.0.
The motivation for this proposal is that at the moment, fedmsg has two
implementations of message signing and verification. The first is based
on M2Crypto and m2ext, while the second is based on cryptography and
The reason there are two implementations is that M2Crypto does not
support Python 3. Python 2 reaches end of life in 30 months. fedmsg is a
dependency of nearly every Infrastructure application and thus it
supporting Python 3 is critical so that we can start the process of
supporting Python 3 in our applications.
In order to provide a Python 3 build of fedmsg for EL7, we need to build
a newer pyOpenSSL. I reviewed the changelogs and from what I can
tell APIs were only extended until pyOpenSSL-17.1.0, at which point
several backwards-incompatible changes were made. I believe we could
safely update to 17.0.0 without breaking applications that depend on it.
I've made a small list of pros and cons to doing this:
* Building a package provided by RHEL's base repository.
* Some risk of breaking applications and libraries using pyOpenSSL.
* We have to maintain it and keep an eye open for any CVEs.
* Even if we do this, fedmsg has to continue to carry the M2Crypto
code unless we want to stop updating fedmsg in EPEL (I'm open to
this, but I'm biased - I really don't want to maintain M2Crypto code).
* Applications have an easier time porting to Python 3. New applications
can seriously consider being Python 3 only.
* A single code path for signing and validating messages for our
What do people think? Is it worth the headache/risk?
Good Morning Everyone,
Earlier today I cut a new release of pagure-dist-git: 0.7.
Here is its changelog:
* Fri Sep 29 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.7-1
- Update to 0.7
- Fix provenpackager access, they also need branch access
- Add support for the custom username in the SSH URL
- Remove caching when querying PDC so that Gitolite ACLs are not generated
with stale data (Matt Prahl)
- Warn the committers if PRs are turned off and there is no readme
- Add a small script dumping the main admins of each project in pagure
It's deployed in stg and so far works fine :)
The output of the cron job referred to in the last line of the changelog can
be found in:
Good Morning Everyone,
In order to fix https://pagure.io/releng/fedora-scm-requests/issue/1078 I would
like to manually apply the following change to pagure on src.fp.o:
@@ -31,7 +31,7 @@ import pagure.lib
STRICT_REGEX = '^[a-zA-Z0-9-_]+$'
TAGS_REGEX = '^[a-zA-Z0-9-_, .]+$'
-PROJECT_NAME_REGEX = '^[a-zA-z0-9_][a-zA-Z0-9-_]*$'
+PROJECT_NAME_REGEX = '^[a-zA-z0-9_][a-zA-Z0-9-_\.]*$'
FALSE_VALUES = ('false', '', False, 'False', 0, '0')
I'll try to find a more elegant/flexible solution for the long term in pagure
itself (like a configuration key).
Good Morning Everyone,
I just cut a new release of pagure: 3.8.
Here is its changelog:
* Fri Sep 29 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 3.8-1
- Update to 3.8
- Fix API documentation for git/branch (Matt Prahl)
- Fix giving a project to someone who already has access (Matth Prahl)
- Add some border to the tables created in README files
- Ask the user to confirm merging a pull-request
- Fix processing status and close_status updates in the SSE
- Fix the URL to the issue used by the SSE JS on tags
- Increase the logging in the milter to help figuring out issues in the future
- Fix the In-Reply-To header when sending notifications
- Fix showing the delete project button
- Fix search issues with a unicode character
- Catch exception raised when accessing the head of the repo
- Fix deleting a project when some of the folder are not used
- Allow viewing a PR when its origin (fork or branch) is gone
- Fix linking to issue or PR in namespaced projects via #<id>
- Make it more obvious that the namespace and the project are different links
- Tell fedmsg to send things with pagure certificates (Patrick Uiterwijk)
- Fix loading ticket templates on namespaced project and extracting their names
- Add a banner on the overview page when the ACLs are being refreshed on the
backend (and thus ssh access may not be entirely functional) (Vivek Anand)
- Update the documentation on how to create pull requests (Clement Verna)
- Add button to refresh external pull requests (Patrick Uiterwijk)
- Add the possibility to get the group members when asking the project info
- Make the PROJECT_NAME_REGEX used in form be configurable
- Adjust the milter to support replying with any email addresses associated
- Allow pagure admin to give a project
Working to updating stg.pagure.io and src.stg.fp.o to it.
Hi, I met up with abompard and Kellin today to solidify a plan to deal
with this issue affecting Fedora:
Here are the notes from the session. It looks like Django makes this
reasonably easy to solve via extending the app. Andrew, the guy
mentioned below, is one of the PDC owners inside Red Hat. I don't
think he's involved in Fedora, which is not an issue; the point is
to keep in touch with him to validate our approach.
* * *
PDC enhancements -- 2017-09-29
Attendees: pfrields, rmarshall, abompard
* What did Rob discover about PDC approach?
* Talked with Andrew, so if we have an app with an API endpoint we can hit with our dataset, that should work well. We can extend the data model the way that we need it.
* Rob's stuff he was looking at was about builder and architecture type, not really related to these issues.
* What can/should Aurélien contribute?
* Looked at PDC source code -- can write the resource containing the necessary models and hooking up an API endpoint
* Should keep in touch with Andrew to get the patch in
* AGREED: abompard will take this work up... target is to get this in so we can use for F28
* Dates can change over time
* E.g. release date slips or the EOL date is changed
* rob: If we can keep a "changelog" of slip dates, that might be helpful. Not sure how to handle this in the API but it would be great if the data model could accommodate somehow.
Paul W. Frields http://paul.frields.org/
gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717
http://redhat.com/ - - - - http://pfrields.fedorapeople.org/
The open source story continues to grow: http://opensource.com
Today I spent a while looking at datagrepper. Our websites build process
calls it and has been failing for a long while now.
We went over some suggestions in
https://pagure.io/fedora-infrastructure/issue/6384 and decided that a db
upgrade could help out, but thats very intrusive so we didn't want to do
it during freeze.
I noticed that haproxy was seeing it stop responding and disabling it
then re-enabling it before nagios alerted and the queries that the
website build was doing were going a long time then returning a 500, so
I started tweaking with the wsgi settings.
Moving it to use processes instead of threads, and adding many more of
them seems to have made it quite stable. The worst case now it takes
60seconds or so to get the last 2 week atomic compose, and usually it's
1 second or so.
My only theory is that all the threads share a db connection somehow so
moving to processes causes it to be able to handle many more at once?
The db connections went from about 20 to about 50, but the load on the
db is down by about 1/2.
Anyhow, the change is below... hopefully it will get us through release.
diff --git a/roles/datagrepper/templates/datagrepper-app.conf
index 6944fb0..86f3ace 100644
@@ -23,7 +23,7 @@ AddOutputFilterByType DEFLATE text/html text/plain
# Static resources for the datagrepper app.
-WSGIDaemonProcess datagrepper user=fedmsg group=fedmsg
maximum-requests=50000 display-name=datagrepper processes=2 threads=2
+WSGIDaemonProcess datagrepper user=fedmsg group=fedmsg
maximum-requests=50000 display-name=datagrepper processes=20 threads=1