New Fedora mirror
by Gray, Thomas
Hello,
I'm Thomas from INFOnline and I want to announce a new Fedora mirror-
server. Currently the mirror is syncing, but we already got a CentOS
and EPEL-mirror up running.
The Coordinates are:
DNS: mirrror.infonline.de
IPv4: 91.215.100.72
IPv6: 2001:67c:2290:1101::1101
Bandwith: 1GBit
Location: Bonn, DE
Kind Regards,
Thomas
6 years, 5 months
Re: NCSU Mirror Account
by Adrian Reber
On Mon, Nov 20, 2017 at 09:07:41AM -0500, Justin Deffenbaugh wrote:
> John is no longer with NCSU. I have created a new account, mirrorsncsu,
> that I'd like to use going forward. If you could add this I'd appreciate it.
I added it as an admin to the NCSU mirror. If you log in you should see
it.
Adrian
6 years, 5 months
NCSU Mirror Account
by Justin Deffenbaugh
Hello, my name is Justin and I've been tasked with updating the Fedora
mirror here at NC State University. I'm in the process of setting up mirror
manager 2, and I need to get an FAS account. Based on config files on the
old server I believe the university has an account already. However, the
people who built our existing mirror server have long since departed and
nobody here has the account info any longer. I'm trying to figure out how
to recover that account info. The forgot password doesn't recognize what I
have for a username.
I'm looking for somebody to work with me to recover the account, or if I
need to just make a new one I can do that as well.
The username I have in my report_mirror.conf file is "NC State University".
Justin
6 years, 5 months
Fedora 27 and Fedora 27 Beta Modular Server
by Mohan Boddu
Hey all,
Fedora 27 and Fedora 27 Beta Modular Server has been staged and should now
be available at
/pub/fedora/linux/releases//test/27_Beta/
/pub/alt/releases//test/27_Beta/
/pub/fedora-secondary/releases//test/27_Beta/
/pub/fedora/linux/modular/releases//test/27_Modular_Beta/
it is 340G(73G for /pub/fedora/linux/modular/releases//test/27_Modular_Beta/)
in pub/fedora,
38G in pub/alt, 310G in pub/fedora-secondary and the rpms are
hardlinked to pub/fedora/linux/development/27/Everything/
Our release is Tuesday the 14th of November 2017, at 14:00 UTC.
Regards,
Fedora Release Engineering.
6 years, 5 months
Changes in EPEL mirror content
by Stephen John Smoogen
As several people have noted, the file directory structure of EPEL has
changed recently. This layout may require changes in both (1) scripts
written with hard-coded locations, and (2) mirrors which were unable
to get daily updates from the main mirrors. While the changes were
communicated in meetings, I did not adequately comprehend their
effects to let mirrors and EPEL users know about it. This meant this
announcement was delayed over two weeks.
===============
What Happened
===============
The updates in the build system were to add new features and make the
release engineering code more manageble. The old release style used by
EPEL in EL-6 and EL-7 was different from how all other releases were
done and caused several problems for the release code and mirrors.
1. Due to all the files of the release being in one directory, any
code which needed to stat (2) the directory caused the server to go
over thousands of files before returning. With EPEL being a large
amount of downloads, this negatively impacted systems. Servers
mirroring the data could find long delays in rsyncing the data
down.
2. The code that generated this was a 'special' case in the Fedora
releng release process which was fragile and tended to cause
problems for updates and releases in both EPEL and the normal
release.
3. The layouts were different from the current Fedora release so that
people grabbing software from multiple places also had to special
case their scripts.
During the updates to the release system with a new version of pungi,
it was decided to remove this special case and have all software
Fedora created laid out in the same structure by the build tools. This
would hopefully make things much more maintainable and improve
performance.
In order to safely transition, there would be a time where the old
files would remain on the server in the old trees and also be
hardlinked to their new location. This was intended to allow for
mirrors to get the files with the minimum amount of bandwidth. However
there were some problems which showed up.
1. As I said before, I didn't grasp that the change was going to
affect EPEL and didn't communicate this to the lists.
2. The transition time for removing the hardlinks was in days versus
weeks. While most mirrors do daily updates, some only do weekly or
monthly rsync's. They missed the hardlinks completely and had to
download data twice.
3. In the usual rule of three, various top level mirrors
(mirrors.kernel.org and some others) had un-related mirroring
problems at the end of October. When these servers caught up with
the new layout, the hardlinked files were gone. This meant that
mirrors taking data from a couple of tier1 sites had large uploads.
=================================
How to deal with current things
=================================
The current layout structure should be 'solid' for the next couple of
years. With the break down of packages into alphabetical subtrees, the
'load' per server should not require a re-ordering in the near
future.
If you have written scripts which downloaded a specific file from the
mirrors, (aka http://dl.fedoraproject.org/pub/archive/epel/5/i386/epel-release-5-4.noar...
or some similar link), you should instead use a stable linked package
like http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
The epel-release packages get updated regularly to get new macros or
other changes so linking to a specific file is very error prone.
Otherwise one should use yum/dnf related commands to get the files
from the mirrors. This is useful for mirror sites which may alter the
directory structure themselves and thus only the repodata is 'safe' to
figure out what to download.
--
Stephen J Smoogen.
6 years, 5 months
EPEL restructuring causing mass download
by Chris Schanzle
Looks like the EPEL 6 and 7 repos transitioned from a flat layout of packages to the Packages/[A-z]/ layout.
Would have been nice to use hardlinks to transition.
Tough on the main mirrors and all their clients to unnecessarily re-download 150+ GB in one day...
6 years, 5 months
