Re: Please hardlink archive/epel and epel
>>>> "ACR" == Anderson, Charles R
<cra(a)wpi.edu> writes:
ACR> I'm not sure whether q-f-m didn't work quite right, whether there
ACR> was a server-side issue (update-archives?), or whether this was
ACR> just all due to timing of cp -l vs. update-archives vs. my q-f-m
ACR> runs, but somehow I received over 1 TB of downloads that were not
ACR> hardlinked.
Here's the issue, put simply:
Currently quick-fedora-mirror will copy hardlinks between rsync modules
as hardlinks if, and only if, it sees the changes in the file lists for
both modules in the same run.
If you hardlink, and then don't update the file lists of both modules
simulteneously, then q-f-m will see a bunch of timestamp updates in one
module on one run, and then in a subsequent run it will see a bunch of
new files in a different module. (The same thing in the reverse order.)
The file lists include a timestamp, which is the newer of mtime and
ctime. Hardlinking will update the timestamp on both files, which is
how it detects that hardlinks were made at all. Then it tells rsync to
transfer both linked files and (due to rsync's '-H' flag) they get
transferred as links.
I have a strategy for working around this, but I haven't implemented it
yet.
ACR> I'm going to look into running quick-fedora-harlink locally
ACR> regularly.
It should always be safe to run. Because links can always be created or
broken on the server in ways that might not be picked up by q-f-m, it's
not a bad idea to run it occasionally. It tries to be "faster" by using
the information in the file lists, but there's still so much stuff in
there that it takes a while. (For more fun, there are even files with
identical names, sizes, and timestamps but not identical content.)
- J<