On Sun, 28 Jul 2019 at 13:17, Anderson, Charles R cra@wpi.edu wrote:
On Sat, Jul 27, 2019 at 10:04:06AM -0400, Stephen John Smoogen wrote:
I did a bunch of copying of packages over to archives at the end of May.
I
do all the copies using cp -l to preserve links. I then ran the update-archives for full file list but then did not see it did not work.
This we got several requests from mirrors to get rid of f28 out of normal space. I confirmed that f28 was copied over and then I synced over f29
and
f30 so I would not be behind in a couple of months. All of these were
also
done with cp -l also and I checked that the hardlink numbers had
increased
on files. I gave the info to Adrian and he let me know that the update-archives was not updated. I then fixed that and again didnt send
any
email about this.
I'm not sure whether q-f-m didn't work quite right, whether there was a server-side issue (update-archives?), or whether this was just all due to timing of cp -l vs. update-archives vs. my q-f-m runs, but somehow I received over 1 TB of downloads that were not hardlinked. It did not finish (I ran out of disk space) so I cleaned that up yesterday, moved the files from .~tmp/* to their final locations, ran a local hardlink.py, and re-ran q-f-m. The hardlink restored the 1+ TB of free space. The files are all staying hardlinked, but due to q-f-m optimizing the rsync runs, that doesn't provide any assurance that the master mirror is as hardlinked as it could be.
To focus on the good parts:
My local hardlink.py run saved at least 600 GB additional space compared to before Friday. I think some of the additional savings comes from internal hardlinking between e.g. Workstation, Everything, Spins, noarch across different arch directories, etc. that one might expect to be already hardlinked, but weren't for some reason.
I'm going to look into running quick-fedora-harlink locally regularly.
So I am going to look at using the hardlink command with a 'do not do this, just tell me what you would do' option first. I will report to the list what I find. This would be with the hardlink we normally ship, but I was wondering if anyone has any new versions they recommend.
Mirror-admin mailing list -- mirror-admin@lists.fedoraproject.org To unsubscribe send an email to mirror-admin-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/mirror-admin@lists.fedoraproje...