About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
7 months, 2 weeks
Proposal to mirror Docker images
by Randy Barlow
During Flock 2016, I had the opportunity to talk with Adam Miller,
Dennis Gilmore, and Pierre-Yves Chibon about the technical challenges
with distributing Docker images with Fedora's extensive mirror network.
These conversations helped me to solidify a proposal for how Fedora
could solve this problem, outlined below.
High level view
===============
In summary, the proposal is to write a patch for the docker client that
will give it the capability to accept metalink responses upon docker
pull operations. We would also need to add support for Docker images to
mirror list and mirror manager. Additionally, we will need a small tool
to pull the content to be mirrored out of a docker registry and write
them to disk in a format that can be mirrored, as well as some Ansible
code to run the tool when there is new content to be mirrored.
Background
==========
The Fedora project wishes to begin distributing new types of content
than it has in the past. One of the types that has been identified as a
goal is the Docker image. Adam Miller has already done the work that
will allow packagers to build Docker images, but we still need a way to
distribute those builds to Fedora's users. Adam Miller's implementation
helpfully drops the builds we want into a Docker registry.
Proposed Changes
================
Mirror List
-----------
Users will be pointing their docker clients at Mirror List when they
docker pull Fedora's Docker images. In order for this to work, we will
need to make two changes to Mirror List so that it can respond to the
docker client properly. The first change is that Mirror List will need
to respond with a special header and a body of "{}" when the docker
client sends a GET request for /v2/. The second change is that it will
need to return a metalink document when the client makes additional
requests so that the clients can be redirected to a list of mirrors
near their locations, just as it does with the dnf client today.
The docker client typically connects to port 5000. We could run a
second instance of Mirror List on port 5000 if we wanted to isolate it
from the current instance. We can also have the docker client pull from
443 as dnf does if we want to keep the deployment simpler.
Mirror Manager
--------------
We will need to make a few changes to Mirror Manager as well. We will
need to provide an interface to allow mirror admins to opt in/out of
mirroring Docker content. We will also need to modify the curler to
detect whether a given mirror is up to date or not. We will need to make
sure that UMDL is updated when content changes.
docker
------
The most significant work required will likely be modifying the docker
client to enable it to properly handle the metalink responses it will be
receiving from Mirror List. When requesting the manifest, it will
receive a metalink document that will give it a priority ordered list of
mirrors. It will need to work through the list in order until it reaches
a mirror that has the correct checksum for the requested manifest. It
will then use that same mirror for the subsequent blob requests.
There is some concern that such a feature would not be accepted by the
upstream docker project. If we were to proceed with this proposal, we
would propose this patch to the upstream Docker project. If upstream
were not willing to accept the feature, we would need to have the Fedora
docker packager carry this patch as a downstream add on.
New Tool
--------
The last piece that is needed is a tool that can create the filesystem
tree that we want to synchronize out to the mirrors. The mirrors only
need to carry manifests and blobs, so the tool needs only to pull these
documents out of the registry that Adam Miller has set up and write them
to disk in a particular structure. For optimization, we could use
hardlinks for blobs that are common across the various images (for
example, the Fedora base blob will be the same in all images) to save
rsync time and mirror disk space.
Additionally, we will need a playbook to run this new tool in response
to fedmsgs. We may be able to use Adam Miller's loopabull project to run
such a playbook at the right times.
Conclusion
==========
Thanks for reading, and please respond with any comments or questions
you have about this proposal. I'm happy to clarify any points further,
and if you have any alternative proposals I'd love to hear those as
well.
6 years, 7 months
[release] pagure: 2.4
by Pierre-Yves Chibon
Good Morning Everyone,
This morning I cut a new release of pagure: 2.4
It had been a while since the last one and I felt we were at a right time for
this. There are more changes coming, some of which will be disruptive so I
wanted to cut a release before them.
Here is the corresponding changelog:
* Wed Aug 31 2016 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 2.4-1
- Update to 2.4
- - [Security] Avoid all html related mimetypes and force the download if any
(CVE-2016-1000037) -- Fixed in 2.3.4 as well
- Redirect the URL to projects <foo>.git to <foo> (Abhishek Goswami)
- Allow creating projects with 40 chars length name on newer pagure instances
- Fix @<user> and #<id> when editing a comment (Eric Barbour)
- Display properly and nicely the ACLs of the API tokens (Lubomír Sedlář)
- Removing html5lib so bleach installation finds what version is best (Tiago M.
Vieira)
- Remove the branchchooser from the repoheader (again) (Ryan Lerch)
- Fix hard-coded urls in the master template
- Made the interaction with the watch button clearer (Ryan Lerch)
- Introduce pagure-ci, a service allowing to integrate pagure with a jenkins
instance (Farhaan Bukhsh and I)
- Accept Close{,s,d} in the same way as Merges and Fixes (Patrick Uiterwijk)
- Avoid showing the 'New PR' button on the overview page is a PR already exists
for this branch, in the main project or a fork (Vivek Anand)
- Fix presenting the readme file and display the readme in the tree page if
there is one in the folder displayed (Ryan Lerch)
- Move the new issue button to be available on every page (AnjaliPardeshi)
- Fix pagure for when an user enters a comment containing #<id> where the id
isn't found in the db
- Make the bootstrap URLs configurable (so that they don't necessarily point to
the Fedora infra) (Farhaan Bukhsh)
- Fix how the web-hook server determine the project and its username
- Replace the login icon with plain text (Ryan Lerch)
- Fix layout in the doc (Farhaan Bukhsh)
- Improve the load_from_disk utility script
- Fix our mardown processor to avoid crashing on #<text> (where we expect #<id>)
- Fix the search for projects with a / in their names
- Fix adding a file to a ticket when running pagure with `local` auth
- Improve the grammar around the allowed prefix in our fake-namespaces (Jason
Tibbitts)
- Implement scanning of attached files for viruses (Patrick Uiterwijk)
- Document how to set-up multiple ssh keys per user (William Moreno Reyes)
- Add display_name and description to groups, and allow editing them
- Add the ability to run the post-receive hook after merging a PR in the UI
- Fix showing the group page even when user management is turned off (Vivek
Anand)
- Make explicit what the separators for tags is (Farhaan Bukhsh)
- Include the word setting with icon (tenstormavi)
- Fix the requirements.txt file (Vivek Anand)
- Cleaned up the topbar a bit (Ryan Lerch)
- Fix location of bottom pagination links on user page (Ryan Lerch)
- Add user's project watch list in index page of the user (Vivek Anand)
- Fix showing the reporter when listing the closed issues (Vivek Anand)
- Fix accessing forks once the main repo has been deleted (Farhaan Bukhsh)
As you can see many changes from many people, thanks to everyone who helped and
worked hard to make this release.
This release has been pushed to staging and will not be pushed to prod before
next week.
Happy hacking,
Pierre
6 years, 9 months
Fedora RelEng Automation
by Adam Miller
Hello all,
I've been doing some work on the Fedora RelEng Automation[0][1]
lately and I've gotten to a point that I have some Ansible playbooks
that I'd like to test but I need to be able to run them somewhere in
the Fedora Infrastructure as they will need to be able to touch
various Release Engineering related systems (at least compose machines
to begin with).
I'd like advisement from the Infrastructure Team on how best to move
forward with this. I'm happy to do the leg work, I just need some
guidance to make sure I'm complying with Infra standards.
Thank you,
-AdamM
[0] - https://pagure.io/releng-automation
[1] - https://fedoraproject.org/wiki/Changes/ReleaseEngineeringAutomationWorkfl...
6 years, 9 months
Mirrormanager doesn't provide F25 x86_64 debuginfo mirror for updates
repository
by Christian Stadelmann
I'm trying to report a bug in a package (gnome-settings-daemon) and it would be useful to have debuginfo for this package. When running `dnf debuginfo-install gnome-settings-daemon`, I'm getting an error that the repomd.xml was not found in metalink. Therefore I manually downloaded the metalink file from https://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f2... (URL from /etc/yum.repos.d/fedora-updates.repo file) and had a look at the file.
The file itself just contains this content:
<metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Wed, 31 Aug 2016 21:05:11 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager">
<!--
# repo = updates-released-debug-f25 arch = x86_64 error: invalid repo or arch
# following repositories are available:
# repo=atomic-21&arch=x86_64
# repo=core-2&arch=i386
# repo=core-2&arch=x86_64
… (more repositories listed) …
# repo=updates-testing-source-fc4&arch=source
# repo=updates-testing-source-fc5&arch=source
# repo=updates-testing-source-fc6&arch=source
-->
</metalink>
but the file does not contain a line like
# repo=updates-released-debug-f25&arch=x86_64
but it contains everything else:
different architectures:
# repo=updates-released-debug-f25&arch=aarch64
# repo=updates-released-debug-f25&arch=ppc64
# repo=updates-released-debug-f25&arch=ppc64le
# repo=updates-released-debug-f25&arch=s390x
different repos:
…
# repo=fedora-debug-25&arch=x86_64
…
# repo=updates-released-f25&arch=x86_64
…
# repo=updates-released-source-f25&arch=source
…
# repo=updates-testing-debug-f25&arch=x86_64
…
the repo I'd need is just missing.
I know of https://apps.fedoraproject.org/packages/mirrormanager/ and bugzilla and http://fedorahosted.org/mirrormanager, but those are for the package respectively its distribution through fedora, not for the infrastructure on mirrors.fedoraproject.org. I hope I'm right here.
6 years, 9 months
Plan for tomorrow's Fedora Infrastructure meeting - 2016-09-01
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2016-09-01 at 18:00 UTC in #fedora-meeting on the freenode network.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
kevin
--
= Introduction =
This shared document is for the next fedora infrastructure meeting.
We will use it over the week before the meeting to gather status and info and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2016-09-01)
#meetingname infrastructure
#topic aloha
#chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson
#topic New folks introductions
= Status / information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need to discuss)
(Please use #info <the thing> - your name)
#topic announcements and information
#info Fedora 25 alpha is out the door! - everyone
#info We are now out of freeze until Beta - everyone
#info new koschei in production on Fedora 24 - mizdebsk
#info openvpn links moved to higher/better encryption - kevin
#info ssl config consolidated, everything using high grade - patrick
#info various small changes made post freeze - kevin
#info
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus or decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)
#topic fedorahosted retirement - kevin
* what to do about lists.fedorahosted.org
* what to do about freemedia
* Other "special" projects?
* When to send announcements
#topic
= Apprentice office hours =
#topic Apprentice Open office hours
Here we will discuss any apprentice questions, try and match up people looking
for things to do with things to do, progress, testing anything like that.
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application or setup
that we have. Just going over what it is, how to contribute, ideas for improvement,
etc. Whoever would like to do this, just add the info in this section. In the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)
#topic Learn about:
= Meeting end stuff =
#topic Open Floor
#endmeeting
6 years, 9 months
Fedora 25 Alpha Freeze now in effect
by Kevin Fenzi
Greetings.
we are now in the infrastructure freeze leading up to the Fedora 25
Alpha release. This is a pre-release freeze.
We do this to ensure that our infrastructure is stable and ready to
release the Fedora 25 Alpha when it's available.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
git clone https://infrastructure.fedoraproject.org/infra/ansible.git
scripts/freezelist -i inventory
Any hosts listed as freezes is frozen until 2016-08-23 (or later if
Alpha slips). Frozen hosts should have no changes made to them
without a sign-off on the change from at least 2 sysadmin-main or
rel-eng members, along with (in most cases) a patch of the exact
change to be made to this list.
Thanks,
kevin
6 years, 9 months
Plan for tomorrow's Fedora Infrastructure meeting - 2016-08-25
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2016-08-25 at 18:00 UTC in #fedora-meeting on the freenode network.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
kevin
--
= Introduction =
This shared document is for the next fedora infrastructure meeting.
We will use it over the week before the meeting to gather status and info and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2016-08-25)
#meetingname infrastructure
#topic aloha
#chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson
#topic New folks introductions
= Status / information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need to discuss)
(Please use #info <the thing> - your name)
#topic announcements and information
#info commops cloud instance fixed up for ssh port changes - kevin
#info copr backend volume fscked, hopefully stable now - kevin
#info Bunch of alpha issues found and sorted, hopefully go this week - kevin
#info freemedia updated for f24 - kevin
#info
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus or decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)
#topic flock workshop followup - kevin
#topic "official" announcement on Trac sunsetting to Pagure - jflory7
#idea To help encourage migration discussion in teams, a post on Community Blog about plans to sunset Trac in favor of Pagure to raise awareness?
= Apprentice office hours =
#topic Apprentice Open office hours
Here we will discuss any apprentice questions, try and match up people looking
for things to do with things to do, progress, testing anything like that.
#info jflory7 would like to pursue becoming a member of sysadmin-badges - interested to know more!
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application or setup
that we have. Just going over what it is, how to contribute, ideas for improvement,
etc. Whoever would like to do this, just add the info in this section. In the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)
#topic Learn about:
= Meeting end stuff =
#topic Open Floor
#endmeeting
6 years, 9 months
