On Thu, Dec 20, 2018 at 11:10:47PM +0000, CLOSE Dave wrote:
I have a script that uses rsync to pull the Fedora repositories
nightly.
If I run it manually, it works flawlessly. But if it runs via cron, I
I encourage you to use Quick Fedora Mirror instead:
https://pagure.io/quick-fedora-mirror.
From the description:
A full rsync of fedora-buffet0 can take hours just to receive the file
list, due to the fact that there are over 11 million files. This has to
happen for every client, every time they want to update, which is
murderous on the download servers and worse on the backend NFS server. It
also slows the propagation of important updates because mirrors simply
can't poll often.
By generating a simple database of file and directory timestamps and
sizes, it becomes easy for the client to determine which files have
changed since the last mirror run and only mirror those files. This also
provides enough information to handle files which have been deleted on the
server and files which are missing on the client. In most situations, it
also allows hardlinks to be copied as hardlinks instead of being
downloaded multiple times.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader