On Sun, 28 Jul 2019 at 13:17, Anderson, Charles R <
cra@wpi.edu> wrote:
On Sat, Jul 27, 2019 at 10:04:06AM -0400, Stephen John Smoogen wrote:
> I did a bunch of copying of packages over to archives at the end of May. I
> do all the copies using cp -l to preserve links. I then ran the
> update-archives for full file list but then did not see it did not work.
>
> This we got several requests from mirrors to get rid of f28 out of normal
> space. I confirmed that f28 was copied over and then I synced over f29 and
> f30 so I would not be behind in a couple of months. All of these were also
> done with cp -l also and I checked that the hardlink numbers had increased
> on files. I gave the info to Adrian and he let me know that the
> update-archives was not updated. I then fixed that and again didnt send any
> email about this.
I'm not sure whether q-f-m didn't work quite right, whether there was
a server-side issue (update-archives?), or whether this was just all
due to timing of cp -l vs. update-archives vs. my q-f-m runs, but
somehow I received over 1 TB of downloads that were not hardlinked.
It did not finish (I ran out of disk space) so I cleaned that up
yesterday, moved the files from .~tmp/* to their final locations, ran
a local hardlink.py, and re-ran q-f-m. The hardlink restored the 1+
TB of free space. The files are all staying hardlinked, but due to
q-f-m optimizing the rsync runs, that doesn't provide any assurance
that the master mirror is as hardlinked as it could be.
To focus on the good parts:
My local hardlink.py run saved at least 600 GB additional space
compared to before Friday. I think some of the additional savings
comes from internal hardlinking between e.g. Workstation, Everything,
Spins, noarch across different arch directories, etc. that one might
expect to be already hardlinked, but weren't for some reason.
I'm going to look into running quick-fedora-harlink locally regularly.
So I am going to look at using the hardlink command with a 'do not do this, just tell me what you would do' option first. I will report to the list what I find. This would be with the hardlink we normally ship, but I was wondering if anyone has any new versions they recommend.
_______________________________________________
Mirror-admin mailing list -- mirror-admin@lists.fedoraproject.org
To unsubscribe send an email to mirror-admin-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/mirror-admin@lists.fedoraproject.org