backup thoughts..

[bruce]

hi.

looking to be able to backup a number of machines running fed (and
possibly other linux flavors)

I'm considering some sort of agent process, where the owner/user can
define the dirs that the owner wants backed up. This kind of approach
would allow us to be able to track when things get backed up, as well
as what's getting backed up.

The downside of this, we don't want to backup all of the OS tree, but
at the same time, if a dev has created apps that toss stuff into the
"OS" tree, it might not get backed up, unless specified...

Thoughts/Comments

thanks

[fedora]

rsync with a wrapper script? I do it this way.

suomi

On 01/09/2014 03:30 PM, bruce wrote:
> hi.
>
> looking to be able to backup a number of machines running fed (and
> possibly other linux flavors)
>
> I'm considering some sort of agent process, where the owner/user can
> define the dirs that the owner wants backed up. This kind of approach
> would allow us to be able to track when things get backed up, as well
> as what's getting backed up.
>
> The downside of this, we don't want to backup all of the OS tree, but
> at the same time, if a dev has created apps that toss stuff into the
> "OS" tree, it might not get backed up, unless specified...
>
> Thoughts/Comments
>
> thanks

[bruce]

you rsync the entire drive?? or just certain trees?

On Thu, Jan 9, 2014 at 9:40 AM, fedora  wrote:
> rsync with a wrapper script? I do it this way.
>
> suomi
>
> On 01/09/2014 03:30 PM, bruce wrote:
> > hi.
> >
> > looking to be able to backup a number of machines running fed (and
> > possibly other linux flavors)
> >
> > I'm considering some sort of agent process, where the owner/user can
> > define the dirs that the owner wants backed up. This kind of approach
> > would allow us to be able to track when things get backed up, as well
> > as what's getting backed up.
> >
> > The downside of this, we don't want to backup all of the OS tree, but
> > at the same time, if a dev has created apps that toss stuff into the
> > "OS" tree, it might not get backed up, unless specified...
> >
> > Thoughts/Comments
> >
> > thanks
>
> --
> users mailing list
> [e-mail]
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
> Have a question? Ask away: http://ask.fedoraproject.org

[fedora]

just certain directories, which may end up in rsyncing the entire drive, 
if necessary.

suomi

On 01/09/2014 03:49 PM, bruce wrote:
> you rsync the entire drive?? or just certain trees?
>
> On Thu, Jan 9, 2014 at 9:40 AM, fedora  wrote:
> > rsync with a wrapper script? I do it this way.
> >
> > suomi
> >
> > On 01/09/2014 03:30 PM, bruce wrote:
> > > hi.
> > >
> > > looking to be able to backup a number of machines running fed (and
> > > possibly other linux flavors)
> > >
> > > I'm considering some sort of agent process, where the owner/user can
> > > define the dirs that the owner wants backed up. This kind of approach
> > > would allow us to be able to track when things get backed up, as well
> > > as what's getting backed up.
> > >
> > > The downside of this, we don't want to backup all of the OS tree, but
> > > at the same time, if a dev has created apps that toss stuff into the
> > > "OS" tree, it might not get backed up, unless specified...
> > >
> > > Thoughts/Comments
> > >
> > > thanks
> >
> > --
> > users mailing list
> > [e-mail]
> > To unsubscribe or change subscription options:
> > https://admin.fedoraproject.org/mailman/listinfo/users
> > Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
> > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
> > Have a question? Ask away: http://ask.fedoraproject.org

[Neal Becker]

Dunno about your specific requirements, but have a look at obnam.  I'm very 
pleased with this for my needs.

[Richard Shaw]

On Thu, Jan 9, 2014 at 8:30 AM, bruce  wrote:
> hi.
>
> looking to be able to backup a number of machines running fed (and
> possibly other linux flavors)
>
> I'm considering some sort of agent process, where the owner/user can
> define the dirs that the owner wants backed up. This kind of approach
> would allow us to be able to track when things get backed up, as well
> as what's getting backed up.
>
> The downside of this, we don't want to backup all of the OS tree, but
> at the same time, if a dev has created apps that toss stuff into the
> "OS" tree, it might not get backed up, unless specified...

I use BackuPC at home. It's clientless (uses the rsync on the host) but
it's designed to be very automated. You can't tell it exactly when to
backup, just a time window. You can specify how many full and incrementals
you want to keep, what directories you want to include/exclude, and even do
a tar (with or without compression) archives out to another disk/location.
It is NOT designed for full system backup/restores and because of the
dependence on rsync, it doesn't handle large files that change frequently
as it has to transfer the whole file again. It does do deduplication so
it's generally effective for home directories, especially where there is a
large number of duplicate files between users.

Richard

[SternData]

On 01/09/2014 10:16 AM, Richard Shaw wrote:
> On Thu, Jan 9, 2014 at 8:30 AM, bruce <[e-mail]
> > wrote:
> hi.
>
> looking to be able to backup a number of machines running fed (and
> possibly other linux flavors)
>
> I'm considering some sort of agent process, where the owner/user can
> define the dirs that the owner wants backed up. This kind of approach
> would allow us to be able to track when things get backed up, as well
> as what's getting backed up.
>
> The downside of this, we don't want to backup all of the OS tree, but
> at the same time, if a dev has created apps that toss stuff into the
> "OS" tree, it might not get backed up, unless specified...
>
>
> I use BackuPC at home. It's clientless (uses the rsync on the host) but
> it's designed to be very automated. You can't tell it exactly when to
> backup, just a time window. You can specify how many full and
> incrementals you want to keep, what directories you want to
> include/exclude, and even do a tar (with or without compression)
> archives out to another disk/location. It is NOT designed for full
> system backup/restores and because of the dependence on rsync, it
> doesn't handle large files that change frequently as it has to transfer
> the whole file again. It does do deduplication so it's generally
> effective for home directories, especially where there is a large number
> of duplicate files between users.
>
> Richard

As long as we're recommending:  I use rdiff_backup to back up my home
directory every morning to the server in the basement.  Once a week, I
use rsync to back up the whole thing to an external USB3 drive via this
script. Because it uses hard links when copying backup.x to backup.x+1,
there's no additonal space needed for files that have not changed, so I
can easily have weeks of backup on the external drive.

#! /bin/sh
mount -t ext4 /dev/sdb1 /mnt/ext-backup
echo last backup was `ls -l /last_backup_ts |cut -d ' ' -f 6-8`
cd /mnt/ext-backup/sds-desk
echo deleting oldest backup
rm -fr backup.F
echo rolling backups
mv backup.E backup.F
mv backup.D backup.E
mv backup.C backup.D
mv backup.B backup.C
mv backup.A backup.B
mv backup.9 backup.A
mv backup.8 backup.9
mv backup.7 backup.8
mv backup.6 backup.7
mv backup.5 backup.6
mv backup.4 backup.5
mv backup.3 backup.4
mv backup.2 backup.3
mv backup.1 backup.2
echo copying backup to backup.1
cp -al backup backup.1
echo updating backup
rsync -va --exclude-from /usr/local/bin/do-backup-excludes --delete /
/mnt/ext-backup/sds-desk/backup/
cd /
df /mnt/ext-backup
echo Wait for external drive to settle
sync
sync
sleep 30
umount /mnt/ext-backup
echo USB drive ejected
touch /last_backup_ts

[Rick Stevens]

On 01/09/2014 06:52 AM, fedora issued this missive:
> just certain directories, which may end up in rsyncing the entire drive,
> if necessary.

bacula works pretty well. You can define directory trees to back up,
exclude lists, backup type (full, incremental, differential), etc. Takes
a bit of work to set up but functions quite well.
> suomi
>
> On 01/09/2014 03:49 PM, bruce wrote:
> > you rsync the entire drive?? or just certain trees?
> >
> > On Thu, Jan 9, 2014 at 9:40 AM, fedora  wrote:
> > > rsync with a wrapper script? I do it this way.
> > >
> > > suomi
> > >
> > > On 01/09/2014 03:30 PM, bruce wrote:
> > > > hi.
> > > >
> > > > looking to be able to backup a number of machines running fed (and
> > > > possibly other linux flavors)
> > > >
> > > > I'm considering some sort of agent process, where the owner/user can
> > > > define the dirs that the owner wants backed up. This kind of approach
> > > > would allow us to be able to track when things get backed up, as well
> > > > as what's getting backed up.
> > > >
> > > > The downside of this, we don't want to backup all of the OS tree, but
> > > > at the same time, if a dev has created apps that toss stuff into the
> > > > "OS" tree, it might not get backed up, unless specified...
> > > >
> > > > Thoughts/Comments

----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital    [e-mail] -
- AIM/Skype: therps2        ICQ: 22643734            Yahoo: origrps2 -
-                                                                    -
-  Any sufficiently advanced technology is indistinguishable from a  -
-                              rigged demo.                          -
----------------------------------------------------------------------

[Patrick O'Callaghan]

On Thu, Jan 9, 2014 at 2:52 PM, fedora  wrote:
> just certain directories, which may end up in rsyncing the entire drive,
> if necessary.

I'll put in a good word for rsnapshot, an rsync wrapper which I've used
happily for several years and has worked well for my needs. Just so people
don't go reinventing the wheel :-)

poc

[Jerry Feldman]

Rsnapshot is a decent wrapper script.

On 01/09/2014 09:40 AM, fedora wrote:
> rsync with a wrapper script? I do it this way.
>
> suomi
>
> On 01/09/2014 03:30 PM, bruce wrote:
> > hi.
> >
> > looking to be able to backup a number of machines running fed (and
> > possibly other linux flavors)
> >
> > I'm considering some sort of agent process, where the owner/user can
> > define the dirs that the owner wants backed up. This kind of approach
> > would allow us to be able to track when things get backed up, as well
> > as what's getting backed up.
> >
> > The downside of this, we don't want to backup all of the OS tree, but
> > at the same time, if a dev has created apps that toss stuff into the
> > "OS" tree, it might not get backed up, unless specified...
> >
> > Thoughts/Comments
> >
> > thanks