we are now in the infrastructure freeze leading up to the Fedora 33
Final release. This is a final release freeze.
We do this to ensure that our infrastructure is stable and ready to
release Fedora 33 when it's available.
You can see a list of hosts that do not freeze by checking out the
ansible repo and running the freezelist script:
ansible/scripts/freezelist -i inventory
Any hosts listed as freezes is frozen until 2020-10-21 (or later if
release slips). Frozen hosts should have no changes made to them without
a sign-off on the change from at least 2 sysadmin-main or rel-eng
members, along with (in most cases) a patch of the exact change to be
made to this list.
You are kindly invited to the meeting:
Fedora Infrastructure on 2020-10-29 from 15:00:00 to 16:00:00 UTC
The meeting will be about:
Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before.
To test authentication with the new AAA system I'd like to deploy a couple
very basic apps that do nothing but auth in staging's openshift. It
shouldn't touch any configuration besides the reverse proxies and the new
project in openshift. And it's staging only.
Is it OK?
On Mon, Oct 26, 2020 at 04:00:06PM +0100, Jan Kasprzak wrote:
> Kevin Fenzi wrote:
> : On Mon, Oct 26, 2020 at 11:12:32AM +0100, Jan Kasprzak wrote:
> : > Hello,
> : >
> : >
> : > Kevin Fenzi wrote:
> : > : As some of you may know, there's been reports of slow sync times for
> : > : content from the master mirrors last week.
> : > : (see: https://pagure.io/fedora-infrastructure/issue/9392 )
> : >
> : > ftp.linux.cz Tier1 mirror admin here. Thanks for the heads up.
> : >
> : > : * Our two mirrors located in other datacenters:
> : > : download-ib01.fedoraproject.org
> : > : and
> : > : download-cc-rdu01.fedoraproject.org
> : >
> : > None of the above contain the fedora-enchilada0 rsync package,
> : > and in the fedora-releases package there is no 33 subdirectory.
> : > Which package should I use for rsyncing?
> : Odd. They do have it defined. Just like the main master mirrors. ;(
> : It could be something to do with them having ipv6 addresses and you
> : coming from a different ipv6 address? We have
> : ftp.linux.cz
> : in rsyncd.conf... so it should reverse lookup the ip and allow it, just
> : like the iad2 masters. ;(
> OK, the reverse lookup would be a problem then. Adding -4 to the
> rsync command line fixes the problem - I can now download from
> Can you add 2001:718:801:230::cd to the ACL? Unfortunately, I have
> many different FQDNs pointing to my server, so PTRs can be inconsistent
> (my fault, I know).
We can. Would it be ok to add it in a few days?
Or do you need it now?
On Mon, Oct 26, 2020 at 11:12:32AM +0100, Jan Kasprzak wrote:
> Kevin Fenzi wrote:
> : As some of you may know, there's been reports of slow sync times for
> : content from the master mirrors last week.
> : (see: https://pagure.io/fedora-infrastructure/issue/9392 )
> ftp.linux.cz Tier1 mirror admin here. Thanks for the heads up.
> : * Our two mirrors located in other datacenters:
> : download-ib01.fedoraproject.org
> : and
> : download-cc-rdu01.fedoraproject.org
> None of the above contain the fedora-enchilada0 rsync package,
> and in the fedora-releases package there is no 33 subdirectory.
> Which package should I use for rsyncing?
Odd. They do have it defined. Just like the main master mirrors. ;(
It could be something to do with them having ipv6 addresses and you
coming from a different ipv6 address? We have
in rsyncd.conf... so it should reverse lookup the ip and allow it, just
like the iad2 masters. ;(
If you can drop by #fedora-admin on irc.freenode.net we can try and
debug it? Or I can just add some hard coded IPv4/IPv6 addresses for you?
> : may provide faster speeds than the dl-tier01.fedoraproject.org mirrors
> : right now depending on your location/routing, so you may want to at
> : least temporarily sync against them.
> So far I have manually hardlinked releases/test/33_Beta tree to
> releases/33, and I am running rsync of
> on top of it. The download speed around 50 Mbit/s.
> Is there a faster way how to get my mirror in sync? Thanks,
Well, fedora-quick-mirror will only copy those specific files you still
need and won't cause our end to have to stat the entire tree so you can
compare it (like bare rsync does). But otherwise that should work...
As some of you may know, there's been reports of slow sync times for
content from the master mirrors last week.
(see: https://pagure.io/fedora-infrastructure/issue/9392 )
This was caused by many routes prefering one of our upstream connections
and saturating it. We are working on increasing BW and rebalancing
routes, but these might take a while to fully take effect.
In the mean time, since Fedora 33 is due out tuesday:
* I have disabled rsync on dl01/02/03. These are our 'non tier1' rsync
servers. Likely we will enable them again tomorrow night to allow
additional mirrors to sync. Note however that if you are not a tier1
mirror, you should sync from one of those:
* Our two mirrors located in other datacenters:
may provide faster speeds than the dl-tier01.fedoraproject.org mirrors
right now depending on your location/routing, so you may want to at
least temporarily sync against them.
* Mirrors are strongly advised to use the https://pagure.io/quick-fedora-mirror/
script to sync if it all possible. This saves you and us lots of I/O and
just syncs the content you need.
Please do check your mirrors and make sure they are syncing up the
Fedora 33 content correctly before tuesdays release.
Below is this week's CPE weekly for week ending 2020-10-25 for both
Fedora & CentOS, and if you want to visit the hackmd link
https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view you can then use the
header bar on your left to skip to Fedora or CentOS updates that
## General Project Updates
We have a CPE Q3 Achievements blog out on the Fedora and CentOS websites
Updated initative timetable can be viewed here
And below are the projects the CPE team are working on for the months
of October, November & December:
* CentOS Stream Phase 4 - Build system services
* Noggin Phase 4 - Data Migration of Fedora & CentOS Accounts, Community testing
* OSBS for aarch64 - this will begin in November
* Fedora Messaging Schemas - this work is continuing from Q3 and is
being worked on part-time
Sent a mail to the devel lists for both Fedora & CentOS with questions
that had answers relating to the topic Accouns & Permissions. It has
been sent to devel-announce(a)fedoraproject.org &
centos-devel(a)centos.org. Here is the link to the hackmd doc I used to
write the email before copying it across to my email to send
## Project Updates
*The below updates are pulled directly from our CPE team call we have
* F33 will be released next week!!
* The team got an F33 nightly compose finished too!
* There is a dashboard available to view the performance of Anitya
* And there is capability to do the same for the packager workflow
pipeline which we are still working on
### Staging Environment
* Build system nearly done - waiting on a firewall change
* New estimated deployment date for Noggin is 29th January 2021.
* The team are working on completing a full staging environment to
deploy Noggin in right now and will hope to have this in place in the
next few weeks
* We also have some members of CentOS working in this team now to help
with the work required for the migration of the CentOS accounts and
data to Noggin
* The teams kanban board where they track their work can be found here
* And we have a project tracker available to be viewed here
### Fedora Messaging Schemas
* This project is worked on on a part time basis as we are
prioritizing completing Noggin first before fully committing to its
* There is a list of applications that require messaging schemas can
be found here https://hackmd.io/@nilsph/H1i8CAbkP/edit
* There is a readme which contains documentation on messaging schemas,
a cookie-cutter template to create the schema and a definition of Done
for writing a schemas
* The board they are working from can be viewed here
## CentOS Updates
* CentOS 7.9.2009 pkgs pushed to CR repo, next step is to install tree
and artifacts such as iso & cloud images.
* Working a lot with the AAA/Noggin team on CentOS account integration
and making good progress
### CentOS Stream
* There are now cloud image updates available for CentOS Stream
* There are some documentation updates here on unshipped packages &
Stream feel free to read up!
## Team Info
### CPE Product Owner Office Hours
IRC office hours are now once per month.Below are the logs from the
most recent meetings and dates for the next ones.
* Next Meeting: 2020-11-12 @ 1300 UTC on #fedora-meeting-1
* Next Meeting: 2020-11-10 @ 1500 UTC on #centos-meeting
The Community Platform Engineering group, or CPE for short, is the Red
Hat team combining IT and release engineering from Fedora and CentOS.
Our goal is to keep core servers and services running and maintained,
build releases, and other strategic tasks that need more dedicated
time than volunteers can give.
See our wiki page here for more
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great week!
Community Platform Engineering Team
Red Hat EMEA
-----BEGIN PGP SIGNED MESSAGE-----
Today, the Fedora Møte team met for one-hour on a Jitsi call. We set up
development goals and began splitting up the backlog for two-week
sprints. A video recording is below:
Some highlights from our meeting:
* Five high-level goals for Møte:
1. Full support for Python 3
2. Convert simple Flask site to a full web app with FE/BE component
3. Mobile support for responsiveness (no refreshing the full D.O.M.)
4. Fully-automated deployment for new updates
5. Continuous integration testing for developers
Some of these things are partially done already, but need additional
follow-up (e.g. Py3 support, CI testing pipeline).
We are using a GitHub Project Board on github.com/fedora-infra to track
and measure our sprints:
Interested in joining the team? Find us in #fedora-mote on Freenode IRC
or join us on Element/Matrix:
Justin W. Flory (he/him)
-----BEGIN PGP SIGNATURE-----
-----END PGP SIGNATURE-----