Qn about rsnapshot/rsync directory filter rules

[Suvayu Ali]

Hi everyone,

I am trying to setup my backups using rsnapshot. I have a rather large
and frequently changing /home/<user>/ that I want to backup. I don't
want to backup the frequently changing directories (e.g. my downalods
directory or temporary work directory). In other words I wanted to
rsnapshot to backup _only_ the directories/files explicitly specified
and skip everythign else.

So I created a file ~/.rsync-filter with include rules like these,

include_/.config/**
include_/dir1/**
include_/.gitconfig
include_/.gitignore
include_/.gitk
include_/dir2/**

Now say I have other files and direcotries like,

~/.oldconfig/
~/dir2_i_dont_want/
~/dir3_i_dont_want/

Calling rsync with -F works but it also backs up the other unwanted
directory and files. How do I exclude them? The problem is I can't write
explicit exclude rules as that would need to be updated every time
something new is created (as I want rsnapshot to _not_ backup anything
by default).

I tried exclude_/* at the end of the filter rules, that made rsnapshot
skip the explicitly mentioned includes like,

include_/dir1/**
include_/dir2/**

The hidden files were backed up just fine though. Any ways of doing what
I want?

[Patrick O'Callaghan]

On Fri, 2011-02-18 at 14:11 -0800, suvayu ali wrote:
> Hi everyone,
>
> I am trying to setup my backups using rsnapshot. I have a rather large
> and frequently changing /home/<user>/ that I want to backup. I don't
> want to backup the frequently changing directories (e.g. my downalods
> directory or temporary work directory). In other words I wanted to
> rsnapshot to backup _only_ the directories/files explicitly specified
> and skip everythign else.
>
> So I created a file ~/.rsync-filter with include rules like these,
>
> include_/.config/**
> include_/dir1/**
> include_/.gitconfig
> include_/.gitignore
> include_/.gitk
> include_/dir2/**
>
> Now say I have other files and direcotries like,
>
> ~/.oldconfig/
> ~/dir2_i_dont_want/
> ~/dir3_i_dont_want/
>
> Calling rsync with -F works but it also backs up the other unwanted
> directory and files. How do I exclude them? The problem is I can't write
> explicit exclude rules as that would need to be updated every time
> something new is created (as I want rsnapshot to _not_ backup anything
> by default).
>
> I tried exclude_/* at the end of the filter rules, that made rsnapshot
> skip the explicitly mentioned includes like,
>
> include_/dir1/**
> include_/dir2/**
>
> The hidden files were backed up just fine though. Any ways of doing what
> I want?

I have the following rule in my /etc/rsnapshot.conf:

exclude_file    /etc/rsnapshot-exclude

where /etc/rsnapshot-exclude contains the following:

.adobe/
.alexandria/
.amaya/
.bittorrent/
.cddb/
.claws-mail/tagsdb/
.cpan/
.dbus/
.gcjwebplugin/
.macromedia/Flash_Player/
.opera/cache4/
.thumbnails/
.waf-*/
.xine/skins/
.gvfs/
.cache/
.ccache/
[Cc]ache/
ccache/
.nautilus/
SVN/
.local/share/evolution
.claws-mail/imapcache
.thunderbird/**/ImapMail
.miro/icon-cache/
.wine/drive_c/windows/temp/
.VirtualBox/
*.o
*.flv
*.mpg
*.mov
*.torrent

There's probably some redundancy there (e.g. I forget why I have cache/
and [Cc]ache/) but it seems to work.

poc

[Suvayu Ali]

Hi Patrick,

Thanks for your response.

On Fri, 18 Feb 2011 21:05:38 -0430
Patrick O'Callaghan  wrote:
> On Fri, 2011-02-18 at 14:11 -0800, suvayu ali wrote:
> > In other words I wanted to rsnapshot to backup _only_ the
> > directories/files explicitly specified and skip everythign else.
>
> I have the following rule in my /etc/rsnapshot.conf:
>
> exclude_file    /etc/rsnapshot-exclude
>
> where /etc/rsnapshot-exclude contains the following:
>
> .adobe/
> .alexandria/
> .amaya/
> ...<snipped>

Yes I follow that, but my problem is this exclude list needs to be
updated everytime I create a new directory (maybe I try out some new
application) or file (which I don't really want to backup). I only want
to backup certian directories/files.
> *.o
> *.flv
> *.mpg
> *.mov
> *.torrent

This however helped me. With this I can consistently ignore certain file
types I don't want to backup ever e.g. downloaded video files or
object/binary files from compilations. I can't do the same for stray
text files I usually end up with when trying to experiment with coding
or looking at log files or temporarily generated list of things with
grep, temporarily downloaded pdf files (I can't ignore all pdfs as I
want to keep papers or notes I am interested in) and many others like
that.

I wish to understand the filter so that I can build the entire rule set
in layers. Something like this,

1. ignore all files
2. add backup_root/dir1
3. add backup_root/some_file

So the subsequent layers overrule (I think logically its an OR) the
previous rule set. This way the backup process will ignore files unless
other wise specified. Is that possible? Or am I asking for something
which is achieved more easily/differently?
> poc

Thanks for any thoughts.

[Tim]

On Sun, 2011-02-20 at 01:20 -0800, Suvayu Ali wrote:
> my problem is this exclude list needs to be updated everytime I create
> a new directory (maybe I try out some new application) or file (which
> I don't really want to backup). I only want to backup certian
> directories/files.

You could structure your directory tree so that you keep everything you
don't want to back-up inside one parent directory, and/or everything
that you do want to get backed-up inside another parent directory.