slightly OT: incremental backup suggestions

[Patrick O'Callaghan]

On Thu, 2014-06-19 at 07:59 -0500, Ranjan Maitra wrote:
> Hi,
> 
> I have decided to also create an incremental backup on my own and was
> wondering what you would recommend. I did some DDG'ing around and came
> up with rdiff-backup. Would you recommend this? (There are
> some more, but this one appears to have an rpm in the fedora
> repositories.) Personally, I am a great fan of rsync but I also wanted a
> setup that would be fast because I would like to run it every hour
> (say). 
> 
> OK, I know this is OT, but it is slightly so because the system used to
> backup are updated F20 systems:-)
> 
> Any suggestions/personal experiences/suggested tweaks/references would
> be greatly appreciated.

I wouldn't call it off-topic for this list.

Personally I use rsnapshot to a local NAS. Some notes:

* Rsnapshot backs up file-by-file. It saves bandwidth by using rsync,
but if a huge file has a single byte change then the sync will be fast
but there will be two copies of the huge file on the backup server.
Consider carefully if you want this to happen with databases or VMs (but
note that sparse files are handled sensibly). There are hooks to use
your own scripts for these cases.

* Rsnapshot keeps a view of a complete snapshot (hence the name) of the
backup-up directories by using hard links on the server. Since
directories can't have arbitrary hard links to them, they have to be
physically copied. This doesn't matter much in practice. (See Apple's
Time Machine which changed filesystem semantics to make directory links
work).

* My first experience of rsnapshot was to run it on my desktop and write
to an NFS-mounted backup space. This works but is slow and inefficient
in network bandwidth. I now run rsnapshot from the NAS and pull data
over SSH. It turns out that this is how rsnapshot is supposed to be
used, but the developers apparently didn't think it worth mentioning in
the docs (I mean, it's so obvious ...) Doing it this way is about an
order of magnitude faster in my case but did require a fair amount of
fracking around on my completely undocumented Iomega NAS.

* Out of the box rsnapshot is focussed on backing up a single client to
a single server. Handling multiple clients requires a little massaging
of the config file. Not a big deal but worth knowing.

poc