i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
OTOH, consider the "git" command, also in /usr/bin:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git
as you can see, there are 116 hardlinks to that executable, pretty much all of them in /usr/libexec/git-core:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-add -rwxr-xr-x. 1 root root 41441 Feb 16 15:03 git-add--interactive -rwxr-xr-x. 3 root root 2273360 Feb 16 15:03 git-am -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-annotate -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-apply -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-archive ... big snip ...
is there a reason that "at" uses symlinks, while "git" installs with hardlinks? one would imagine that it would be the other way around -- given that the "at" variations are all in the same directory, hardlinks would seem to be the better choice since there is no possibility of crossing filesystem boundaries.
also, what if i manually partitioned while installing, and put /usr/bin and /usr/libexec in separate partitions? (yes, i realize that's not the standard these days -- all of /usr defaults to a single partition -- but i'm assuming i still have the freedom to do something that unwise.) would git then switch to symlinks to support all those command name variants?
just curious about how decisions like this are made WRT to packaging.
rday
On 03/06/18 20:34, Robert P. J. Day wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
I may be out of line here. It is just that I too am curious.
A short while back you asked a question about "user profiles" and you did mention it was due to being "handed a linux course to teach for a new client". Is this part of the same effort?
On Tue, 6 Mar 2018, Ed Greshko wrote:
On 03/06/18 20:34, Robert P. J. Day wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
I may be out of line here. It is just that I too am curious.
A short while back you asked a question about "user profiles" and you did mention it was due to being "handed a linux course to teach for a new client". Is this part of the same effort?
these recent questions are definitely inspired by the 5-day course i taught last week, which ostensibly prepares students to write a couple exams to get their LPIC-1 certification through comptia, after writing and passing two exams:
https://certification.comptia.org/certifications/linux#examdetails
a lot of the material was stuff i was intimately familiar with, but every so often, such as when i was presenting the section on links, i'd suggest that links were used to support alternate commands for closely-related commands, and i'd show, say, what was under /usr/bin, and i'd suddenly get caught up short as i noticed there were both symlinks and hardlinks, and i had no explanation as to the rationale for the choice between the two, because it had never occurred to me to look that closely before. (in my defense, this was on centos 7, which i don't normally use, but i'm seeing a lot of the same on fedora 27, so the questions still apply.)
hence, the spectacularly pedantic questions the last couple of days -- there's nothing like trying to explain what you *think* you know everything about to relative linux newbies, only to trip over something trivial and mutter, "huh ... i don't recall ever seeing *that* before."
so there's probably more admittedly trivial questions coming, but only because i'm being forced to look more closely at things than i have in a long time.
rday
On 03/06/18 22:30, Robert P. J. Day wrote:
so there's probably more admittedly trivial questions coming, but only because i'm being forced to look more closely at things than i have in a long time.
OK. Then I feel compelled to ask the following in the spirit of full-disclosure.
In answering these queries are we contributing information to your knowledge base that will be incorporated into a course for which you'll be compensated?
On 03/06/2018 06:34 AM, Robert P. J. Day wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
OTOH, consider the "git" command, also in /usr/bin:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git
as you can see, there are 116 hardlinks to that executable, pretty much all of them in /usr/libexec/git-core:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-add -rwxr-xr-x. 1 root root 41441 Feb 16 15:03 git-add--interactive -rwxr-xr-x. 3 root root 2273360 Feb 16 15:03 git-am -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-annotate -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-apply -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-archive ... big snip ...
is there a reason that "at" uses symlinks, while "git" installs with hardlinks? one would imagine that it would be the other way around -- given that the "at" variations are all in the same directory, hardlinks would seem to be the better choice since there is no possibility of crossing filesystem boundaries.
In the case of "git" there's the matter of using 1 inode vs. 116 inodes.
On Tue, 6 Mar 2018, Robert Nichols wrote:
On 03/06/2018 06:34 AM, Robert P. J. Day wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
OTOH, consider the "git" command, also in /usr/bin:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git
as you can see, there are 116 hardlinks to that executable, pretty much all of them in /usr/libexec/git-core:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-add -rwxr-xr-x. 1 root root 41441 Feb 16 15:03 git-add--interactive -rwxr-xr-x. 3 root root 2273360 Feb 16 15:03 git-am -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-annotate -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-apply -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-archive ... big snip ...
is there a reason that "at" uses symlinks, while "git" installs with hardlinks? one would imagine that it would be the other way around -- given that the "at" variations are all in the same directory, hardlinks would seem to be the better choice since there is no possibility of crossing filesystem boundaries.
In the case of "git" there's the matter of using 1 inode vs. 116 inodes.
sure, and that's admittedly an extreme case, but the same logic applies to saving 2 or 3 inodes.
rday
Robert P. J. Day wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
I would bet that often the use of symlink or hardlink is part of the upstream install scripts. Unless there is a strong reason to change that, the packaging should just follow upstream.
OTOH, consider the "git" command, also in /usr/bin:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git
as you can see, there are 116 hardlinks to that executable, pretty much all of them in /usr/libexec/git-core:
-rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-add -rwxr-xr-x. 1 root root 41441 Feb 16 15:03 git-add--interactive -rwxr-xr-x. 3 root root 2273360 Feb 16 15:03 git-am -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-annotate -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-apply -rwxr-xr-x. 116 root root 2273360 Feb 16 15:03 git-archive ... big snip ...
is there a reason that "at" uses symlinks, while "git" installs with hardlinks? one would imagine that it would be the other way around -- given that the "at" variations are all in the same directory, hardlinks would seem to be the better choice since there is no possibility of crossing filesystem boundaries.
also, what if i manually partitioned while installing, and put /usr/bin and /usr/libexec in separate partitions? (yes, i realize that's not the standard these days -- all of /usr defaults to a single partition -- but i'm assuming i still have the freedom to do something that unwise.) would git then switch to symlinks to support all those command name variants?
just curious about how decisions like this are made WRT to packaging.
The git package uses hardlinks and includes some duplication of binaries precisely to avoid cross-directory hardlinks. That allows /usr/bin and /usr/libexec to be mounted on different file systems. It's not all that common to do that, I'm sure.
But rpmlint warns about such cross-directory hardlinks. While working on the git packaging I used the support in upstream git's Makefile (NO_CROSS_DIRECTORY_HARDLINKS):
https://src.fedoraproject.org/rpms/git/c/6ef5f1f7232
That change was made in the master (rawhide) branch. It will be part of Fedora 28. It's not a change worth applying to other releases by itself, so I don't expect to see it pushed to the current Fedora 26/27 releases.
Ideally such changes are reasonably described in the commit message. I strive to write detailed commit messages so future maintainers or anyone curious can see why a given change was made. Not every packager does this, of course. And I'm sure that despite my best efforts, I might not always describe the reasoning for a change in a way that answers all the questions someone may have in the future.
I think it's worth poking through the git history of the packages you're curious about. There may be some answers about the others in the commit logs. The git repositories for all Fedora packages can be found at:
https://src.fedoraproject.org/
On Tue, 6 Mar 2018, Todd Zullinger wrote:
... snip ...
The git package uses hardlinks and includes some duplication of binaries precisely to avoid cross-directory hardlinks. That allows /usr/bin and /usr/libexec to be mounted on different file systems.
i was about to point out that, in fact, the git-core package *does* hardlink across directories (that is, /usr/bin/git has a ton of hardlinks from /usr/libexec/git-core ...
It's not all that common to do that, I'm sure.
But rpmlint warns about such cross-directory hardlinks. While working on the git packaging I used the support in upstream git's Makefile (NO_CROSS_DIRECTORY_HARDLINKS):
https://src.fedoraproject.org/rpms/git/c/6ef5f1f7232That change was made in the master (rawhide) branch. It will be part of Fedora 28. It's not a change worth applying to other releases by itself, so I don't expect to see it pushed to the current Fedora 26/27 releases.
... ah, so the replacement of those cross-directory hardlinks with symlinks will happen in F28, is that what you're saying?
rday
On 6 March 2018 at 14:34, Robert P. J. Day rpjday@crashcourse.ca wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
I am not an expert, some commands act differently when called differently; so executing /bin/atq would make the 'at' binary behave differently than when it's executed as plain 'at'.
The same logic applies to bash, /bin/sh is a symlink to /bin/bash, but when bash is invoked as sh it acts differently than when invoked as 'bash'. Have a look at the bash manual page for more info.
On Wed, 7 Mar 2018, Ahmad Samir wrote:
On 6 March 2018 at 14:34, Robert P. J. Day rpjday@crashcourse.ca wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
I am not an expert, some commands act differently when called differently; so executing /bin/atq would make the 'at' binary behave differently than when it's executed as plain 'at'.
The same logic applies to bash, /bin/sh is a symlink to /bin/bash, but when bash is invoked as sh it acts differently than when invoked as 'bash'. Have a look at the bash manual page for more info.
yes, i'm aware of this, and i'm pretty sure whether a hardlink or symlink is used would make no difference.
rday
Robert P. J. Day wrote:
... ah, so the replacement of those cross-directory hardlinks with symlinks will happen in F28, is that what you're saying?
The change will be in F28, yes. The few files in /usr/bin are simply copied, not symlinked. Within /usr/bin, the identical files are hardlinked to each other.
On 9/3/18 6:13 am, Todd Zullinger wrote:
Robert P. J. Day wrote:
... ah, so the replacement of those cross-directory hardlinks with symlinks will happen in F28, is that what you're saying?
The change will be in F28, yes. The few files in /usr/bin are simply copied, not symlinked. Within /usr/bin, the identical files are hardlinked to each other.
I haven't been keeping up with this thread prior to now so I apologize if I'm covering old ground or have misinterpreted what this thread is saying. It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents, whereas the same doesn't happen with symlinks. Are you saying that with F28 hardlinks are going to be replaced by symlinks, so that the hardlink functionality no longer works, or are you saying that symlink functionality is being changed to function the same as hardlinks, hence we lose the existing symlink functionality?
regards,
Steve
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org
On Fri, 2018-03-09 at 07:59 +1100, Stephen Morris wrote:
It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents
What you describe here is linking, not copying. Copying always produces an apparently independent file ('Apparently' because on Copy-On-Write filesystems they two may actually share disk blocks until one of them changes, but that is *not* the same as linking).
poc
On 03/08/2018 12:59 PM, Stephen Morris wrote:
On 9/3/18 6:13 am, Todd Zullinger wrote:
Robert P. J. Day wrote:
... ah, so the replacement of those cross-directory hardlinks with symlinks will happen in F28, is that what you're saying?
The change will be in F28, yes. The few files in /usr/bin are simply copied, not symlinked. Within /usr/bin, the identical files are hardlinked to each other.
I haven't been keeping up with this thread prior to now so I apologize if I'm covering old ground or have misinterpreted what this thread is saying. It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents, whereas the same doesn't happen with symlinks. Are you saying that with F28 hardlinks are going to be replaced by symlinks, so that the hardlink functionality no longer works, or are you saying that symlink functionality is being changed to function the same as hardlinks, hence we lose the existing symlink functionality?
You definitely haven't been keeping up. :-)
This is a discussion about a specific package (git). It has a lot of identical binary files with different names in two different directories. Currently these are all hard linked to each other. What will change is that the cross-directory hardlinks will be removed.
On Thu, 8 Mar 2018, Samuel Sieb wrote:
On 03/08/2018 12:59 PM, Stephen Morris wrote:
On 9/3/18 6:13 am, Todd Zullinger wrote:
Robert P. J. Day wrote:
... ah, so the replacement of those cross-directory hardlinks with symlinks will happen in F28, is that what you're saying?
The change will be in F28, yes. The few files in /usr/bin are simply copied, not symlinked. Within /usr/bin, the identical files are hardlinked to each other.
I haven't been keeping up with this thread prior to now so I apologize if I'm covering old ground or have misinterpreted what this thread is saying. It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents, whereas the same doesn't happen with symlinks. Are you saying that with F28 hardlinks are going to be replaced by symlinks, so that the hardlink functionality no longer works, or are you saying that symlink functionality is being changed to function the same as hardlinks, hence we lose the existing symlink functionality?
You definitely haven't been keeping up. :-)
This is a discussion about a specific package (git). It has a lot of identical binary files with different names in two different directories. Currently these are all hard linked to each other. What will change is that the cross-directory hardlinks will be removed.
that was my understanding -- as long as the files are within precisely the same directory, hard links could still be used, but any cross-directory links (even if within the same filesystem) will use symlinks. is that about right?
rday
On 03/09/2018 01:27 AM, Robert P. J. Day wrote:
that was my understanding -- as long as the files are within precisely the same directory, hard links could still be used, but any cross-directory links (even if within the same filesystem) will use symlinks. is that about right?
From what you were saying, it sounded like you thought it was a system-wide change for how linking worked.
Hard links can be used between directories on the same file system, but for a package install, since you don't know how the partitions will be configured, it's safer to to use symlinks if not in the same directory. That's what is being changed for this package.
On 9/3/18 9:11 am, Patrick O'Callaghan wrote:
On Fri, 2018-03-09 at 07:59 +1100, Stephen Morris wrote:
It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents
What you describe here is linking, not copying. Copying always produces an apparently independent file ('Apparently' because on Copy-On-Write filesystems they two may actually share disk blocks until one of them changes, but that is *not* the same as linking).
No, what I was mentioning here is what I have read as standard linux functionality with copying, when a file is copied, and it doesn't matter where to, rather than create a 2nd copy of the file, the "copy" is created as a hard link to the original file, for storage efficiency, and then when one of the files is updated the hardlink is broken and both files become physical.
regards,
Steve
poc _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org
On Sat, 10 Mar 2018, Stephen Morris wrote:
On 9/3/18 9:11 am, Patrick O'Callaghan wrote:
On Fri, 2018-03-09 at 07:59 +1100, Stephen Morris wrote:
It is my understanding that currently when a file copied to any location, a physical copy is not produced, the copy is a hardlink to the original file, until such time as one of the "copies" is changed and then both become physical files with one file reflecting the pre-change contents
What you describe here is linking, not copying. Copying always produces an apparently independent file ('Apparently' because on Copy-On-Write filesystems they two may actually share disk blocks until one of them changes, but that is *not* the same as linking).
No, what I was mentioning here is what I have read as standard linux functionality with copying, when a file is copied, and it doesn't matter where to, rather than create a 2nd copy of the file, the "copy" is created as a hard link to the original file, for storage efficiency, and then when one of the files is updated the hardlink is broken and both files become physical.
AFAIK, "copying" in unix/linux has never worked that way; if you can provide a link that describes it that way, that would be interesting.
rday
Samuel Sieb wrote:
On 03/09/2018 01:27 AM, Robert P. J. Day wrote:
that was my understanding -- as long as the files are within precisely the same directory, hard links could still be used, but any cross-directory links (even if within the same filesystem) will use symlinks. is that about right?
From what you were saying, it sounded like you thought it was a system-wide change for how linking worked.
Hard links can be used between directories on the same file system, but for a package install, since you don't know how the partitions will be configured, it's safer to to use symlinks if not in the same directory. That's what is being changed for this package.
In the git package, there aren't symlinks. Within /usr/bin, the git binaries which are identical are hardlinked to each other. And separately, within /usr/libexec/git-core, the git binaries which are identical are hardlinked to each other.
For anyone really curious about the finer details on this, I would point to the upstream git Makefile¹ and the Fedora git spec file².
¹ https://github.com/git/git/blob/master/Makefile ² https://src.fedoraproject.org/rpms/git/blob/master/f/git.spec
On Sat, 2018-03-10 at 09:57 +1100, Stephen Morris wrote:
No, what I was mentioning here is what I have read as standard linux functionality with copying, when a file is copied, and it doesn't matter where to, rather than create a 2nd copy of the file, the "copy" is created as a hard link to the original file, for storage efficiency, and then when one of the files is updated the hardlink is broken and both files become physical.
Considering all the grinding noises my hard drive makes if I copy a file, I wouldn't have said it works in that way.
On 03/09/2018 03:06 PM, Todd Zullinger wrote:
Samuel Sieb wrote: In the git package, there aren't symlinks. Within /usr/bin, the git binaries which are identical are hardlinked to each other. And separately, within /usr/libexec/git-core, the git binaries which are identical are hardlinked to each other.
But the split between /usr/bin and /usr/libexec is a new thing in F28 according to your earlier email, right?
Samuel Sieb wrote:
On 03/09/2018 03:06 PM, Todd Zullinger wrote:
In the git package, there aren't symlinks. Within /usr/bin, the git binaries which are identical are hardlinked to each other. And separately, within /usr/libexec/git-core, the git binaries which are identical are hardlinked to each other.
But the split between /usr/bin and /usr/libexec is a new thing in F28 according to your earlier email, right?
Yeah. In current releases the files are hardlinked across /usr/bin and /usr/libexec. I've never seen any bug reports about that causing actual problems. I made the change after noticing the cross-directory hardlink warning from rpmlint.
It should really be an unnoticable change to nearly everyone, apart from those rare people that want to mount /usr/bin and /usr/libexec on different file systems
On Fri, Mar 9, 2018 at 2:57 PM, Stephen Morris samorris@netspace.net.au wrote:
No, what I was mentioning here is what I have read as standard linux functionality with copying, when a file is copied, and it doesn't matter where to, rather than create a 2nd copy of the file, the "copy" is created as a hard link to the original file, for storage efficiency, and then when one of the files is updated the hardlink is broken and both files become physical.
The only process I can think of that works even remotely like that is rsnapshot (and similar backup systems). In those setups, rsync will create new files for the first backup. On the second backup, it creates hard links to all of the files in the set, first, and then runs rsync to refresh the directory holding the newest backup. When rsync runs, it does not modify files in place, it creates a new file and merges data from the local file and data from the remote file in order to save bandwidth, according to its specialized algorithm. When it's done, it moves the new file into place, atomically updating the path in the backup.
As you describe, rsnsapshot uses hard links to conserve disk space and breaks links when it updates files. My guess is that at some point in the past, you read about rsnapshot or a similar backup system, and later remembered their mode of operation as standard functionality.
On 11/3/18 4:35 am, Gordon Messmer wrote:
On Fri, Mar 9, 2018 at 2:57 PM, Stephen Morris samorris@netspace.net.au wrote:
No, what I was mentioning here is what I have read as standard linux functionality with copying, when a file is copied, and it doesn't matter where to, rather than create a 2nd copy of the file, the "copy" is created as a hard link to the original file, for storage efficiency, and then when one of the files is updated the hardlink is broken and both files become physical.
The only process I can think of that works even remotely like that is rsnapshot (and similar backup systems). In those setups, rsync will create new files for the first backup. On the second backup, it creates hard links to all of the files in the set, first, and then runs rsync to refresh the directory holding the newest backup. When rsync runs, it does not modify files in place, it creates a new file and merges data from the local file and data from the remote file in order to save bandwidth, according to its specialized algorithm. When it's done, it moves the new file into place, atomically updating the path in the backup.
As you describe, rsnsapshot uses hard links to conserve disk space and breaks links when it updates files. My guess is that at some point in the past, you read about rsnapshot or a similar backup system, and later remembered their mode of operation as standard functionality.
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
regards,
Steve
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org
On 03/11/18 09:24, Stephen Morris wrote:
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
From "man cp"
-l, --link hard link files instead of copying
-s, --symbolic-link make symbolic links instead of copying
[egreshko@meimei tmp]$ ll -i source 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
[egreshko@meimei tmp]$ cp -s source source-s
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
[egreshko@meimei tmp]$ cp -l source source-h
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
So, this is probably what you're recalling
On 11/3/18 1:09 pm, Ed Greshko wrote:
On 03/11/18 09:24, Stephen Morris wrote:
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
From "man cp"
-l, --link hard link files instead of copying
-s, --symbolic-link make symbolic links instead of copying
[egreshko@meimei tmp]$ ll -i source 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
[egreshko@meimei tmp]$ cp -s source source-s
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
[egreshko@meimei tmp]$ cp -l source source-h
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
So, this is probably what you're recalling
It could be Ed, I remember the documentation differently, so I could be not remembering it correctly, so I'll keep quiet now.
regards,
Steve
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org
On Sun, 2018-03-11 at 19:59 +1100, Stephen Morris wrote:
On 11/3/18 1:09 pm, Ed Greshko wrote:
On 03/11/18 09:24, Stephen Morris wrote:
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
From "man cp"
-l, --link hard link files instead of copying -s, --symbolic-link make symbolic links instead of copying[egreshko@meimei tmp]$ ll -i source 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source
[egreshko@meimei tmp]$ cp -s source source-s
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 1 egreshko egreshko 14 Mar 11 10:06 source 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
[egreshko@meimei tmp]$ cp -l source source-h
[egreshko@meimei tmp]$ ll -i source* 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source 11574505 -rw-rw-r--. 2 egreshko egreshko 14 Mar 11 10:06 source-h 11573763 lrwxrwxrwx. 1 egreshko egreshko 6 Mar 11 10:07 source-s -> source
So, this is probably what you're recalling
It could be Ed, I remember the documentation differently, so I could be not remembering it correctly, so I'll keep quiet now.
The original version of 'cp' (by which I mean the version going back to the early days of Unix) didn't have any options. It simply copied the file contents to a different file. There was no question of using links, hard or soft (in fact symbolic links hadn't been invented).
poc
On 03/10/2018 05:24 PM, Stephen Morris wrote:
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
I'd forgotten the other system that works as you described: reflink.
"cp" supports a --reflink option, and on a few filesystems that implement it (btrfs, OCFS2, maybe xfs and zfs, I'm unclear on their status), you can create a CoW copy of a file.
Again, not standard behavior, not widely implemented, and relatively new, but this might be what you were thinking of.
On Sun, 11 Mar 2018, Gordon Messmer wrote:
On 03/10/2018 05:24 PM, Stephen Morris wrote:
I can't find the documentation any more, but I have found documentation on how to use copy the create the target as a hard link or as a soft link. It is possible I have incorrectly remembered what I had read, or it is possible over time that the standard copy functionality has changed and now you have to explicitly specify that you want that functionality.
I'd forgotten the other system that works as you described: reflink.
"cp" supports a --reflink option, and on a few filesystems that implement it (btrfs, OCFS2, maybe xfs and zfs, I'm unclear on their status), you can create a CoW copy of a file.
Again, not standard behavior, not widely implemented, and relatively new, but this might be what you were thinking of.
ok, that's cool, i had no idea that even existed.
rday
On 6 March 2018 at 10:45, Ed Greshko ed.greshko@greshko.com wrote:
On 03/06/18 22:30, Robert P. J. Day wrote:
so there's probably more admittedly trivial questions coming, but only because i'm being forced to look more closely at things than i have in a long time.
OK. Then I feel compelled to ask the following in the spirit of full-disclosure.
In answering these queries are we contributing information to your knowledge base that will be incorporated into a course for which you'll be compensated?
Actually, the information could be incorporated into courses by people lurking on the list or maybe finding the thread using google. Good questions are in themselves useful contributions, so are welcome unless they appear to come from a homework assignment.
If simple questions can be answered with a simple google query then why are there so many of them?
Maybe because people have learned that answers to google queries are often incorrect. If there is disagreement over the answer, forums let you see the reasoning behind different proposed answers. Some of the discussions in this forum have been educational.
In the interest of a full discussion, there is: https://unix.stackexchange.com/questions/192015/links-in-usr-bin-or-usr-loca...
On 7 March 2018 at 12:34, Robert P. J. Day rpjday@crashcourse.ca wrote:
On Wed, 7 Mar 2018, Ahmad Samir wrote:
On 6 March 2018 at 14:34, Robert P. J. Day rpjday@crashcourse.ca wrote:
i'm curious about RH packaging policy that dictates that some command variants are packaged for fedora to install with symlinks and others with hardlinks.
trivial example in /usr/bin on my fedora 27 system:
-rwsr-xr-x. 1 root root 52984 Aug 2 2017 at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atq -> at lrwxrwxrwx. 1 root root 2 Aug 2 2017 atrm -> at
so even though all of those "commands" are in the very same directory, atq and atrm are supported via symlinks, not hardlinks.
I am not an expert, some commands act differently when called differently; so executing /bin/atq would make the 'at' binary behave differently than when it's executed as plain 'at'.
The same logic applies to bash, /bin/sh is a symlink to /bin/bash, but when bash is invoked as sh it acts differently than when invoked as 'bash'. Have a look at the bash manual page for more info.
yes, i'm aware of this, and i'm pretty sure whether a hardlink or symlink is used would make no difference.
rday
You are right, I missed the bit about hardlinks. Sorry for the noise.
-
Ahmad Samir -
Ed Greshko -
George N. White III -
Gordon Messmer -
Patrick O'Callaghan -
Robert Nichols -
Robert P. J. Day -
Samuel Sieb -
Stephen Morris -
Tim -
Todd Zullinger