Filesystem format for external hard disk

[Paul Smith]

On Thu, May 31, 2012 at 4:43 PM, Jeff Gipson <jeffagipson at gmail.com> wrote:
>> Where do you place your backups, Jeff? :-)
>
> Well, I just started using SpiderOak. Jury's still out. Their
> deduplication and compression ratio seems pretty good (185GiB on my disk
> becomes 145GiB in their system), and they support file versioning, but
> don't seem to have a point-in-time restore feature. I think I can create
> a script to do that, but I liked my old rsync backups better.
>
> Also, you either have to run their GUI client *all* the time (or you
> guessed it, no scheduled backups run) OR you have to run the client via
> Cron in a headless sort of mode, when means (you guessed it... no GUI).
>
> You can't run multiple instances of the client.
>
> This limitation hasn't been an issue for me. I ran the GUI to configure
> my backup selections, then quit the GUI and use cron to run the backups
> on my schedule; But I can see this being a show-stopper for some people.
>
> If I hadn't just bought a year in advance, I'd probably run the old
> rsync script that I used to use which send the data over a VPN (or ssh)
> link to a remote system. It created a hardlink structure which was handy
> for point-in-time restores, i.e. on the remote system:
>
>  backups/2012-01-01/DriveC
>  backups/2012-01-01/DriveD
>  backups/2012-01-02/DriveC
>  backups/2012-01-02/DriveD
>
>  etc...
>
> With the exception of the first backup, all backups were incremental.
> The result was that you could enter the folder for whatever day you
> wanted and see all the files as they were on that date, not just the
> ones that changed and were backed up. The rsync command was a monster:
>
>  /usr/local/bin/rsync -az --stats --timeout=300 --progress \
>                       --delete --delete-excluded --exclude-from="$6" \
>                       --log-format="%t %f (%l/%b)" \
>                       --link-dest="$DESTRT$PREFIX_TAG-Previous" "$RSYNCSHAR"
> # then
>  ln -sf "$DEST" "$PREFIX_TAG-Previous"
>
> Of course the script set many of the other variables. I'd probably write
> this script slightly different now (this script was written 8 years ago)
> but the principle is the same.
>
>  1) Do an initial backup with rsync an set the "previous" symlink to
>  point to the destination folder
>  2) Next day, create a new date-code-based destination folder, backup
>  using rsync against the previous destination folder (see --link-dest)
>  3) Update the "previous" symlink to point to the most recent backup
>  4) repeat steps 2-3
>
> My script was also kinda ugly, but it ran for years, and with the links
> neatly arranges as described above, I was able to configure an apache
> server to allow other users to browse and download old versions on their
> own.
>
> It's not perfect, I'm sure, but here's the whole script in context (for
> educational purposes)
>
> http://fpaste.org/ry7G/  <- note This paste will self-destruct by
> tomorrow.

Thanks, Jeff, for your so detailed and helpful answer!

Paul