optimesing NFS Server for e.g. remote Backup

[John Summerfied]

Tony Molloy wrote:
> On Thursday 16 February 2006 07:45, Götz Reinicke wrote:
> 
>>Hi,
>>
>>I have some RHEL4 servers and want to set up a central backup system
>>with a "big" harddisk. I found a very nice shell-script, doing
>>incremental backups using tar.
>>
> 
> 
>>Any ideas what coul be wrong? The Backups are about 10 GB +- up to 100
>>GB in the future.
> 
> 
> With that amount of data to backup don't use tar, use rsync instead. Rsync 
> is much faster as it only copies the parts of files which have changed. 
> An excellent backup system using rsync to try is BackupPC                
> <  backuppc.sourceforge.net >. It does full and incremental backups using 
> rsync.

Some time ago, I tried backing up our server over the Internet using 
rsync, and even after discussion with the developers and updating to the 
latest version, it was way too slow.

It took enormous amounts of RAM and swap (fortunately, depite making 
swap several times RAM it didn't induce thrashing), and never completed 
in a reasonable time.

Now, I have a devious script that creates a compressed ISO using 
mkzftree and mkisofs and then uses rsync to update a remote copy over 
ADSL. The longest time to update the remote copy is a whisker over the 
hour which we think entirely reasonable.

It takes around five minutes to copy over the local LAN.

Here is the script: watch for line-wrap.
mail:~# cat /root/bin/bu.image
#!/bin/bash -e
RSO="--recursive --links --hard-links --perms --owner --group --devices 
--times"
RSO="${RSO} --sparse --one-file-system --rsh=/usr/bin/ssh --delete "
RSO="${RSO} --delete-excluded --relative --stats "
RSO="${RSO} --numeric-ids --timeout=7200 ${EXC}"
trap '/etc/init.d/postfix start' EXIT
/etc/init.d/postfix stop
date
[ -d /mnt/backup ] || mkdir /mnt/backup
Image=/var/tmp/backup.img
umount /mnt/backup || :
rm -fr ${Image} /var/tmp/backup
dd if=/dev/zero of=${Image} count=0 bs=1024 \
         seek=$((5*1024*1024))
ls -shog ${Image}
mke2fs -Fq ${Image}
mount -o loop ${Image} /mnt/backup/
sfdisk >/etc/disktab -d /dev/hda
tar clC / --exclude=backup.img --exclude=/tmp --exclude=/mnt\/* 
--exclude=/var/lock --exclude=swapfil\* \
         --exclude=/var/autofs --exclude=lost+found --exclude=/var/tmp 
--exclude=/var/local --exclude=squid-cache \
         --exclude=/var/spool/cyrus/mail-backup \
         / /boot /home  /var \
         | buffer -m $((2*1024*1024)) -p 75\
         | tar xpC /mnt/backup || { df -h ; exit ; }
ls -go /mnt/backup
rm ${Image}
/etc/init.d/postfix start && trap -
mkzftree --one-filesystem /mnt/backup/ /var/tmp/backup
uptime
df -h
umount /mnt/backup/
mkisofs -R -z -quiet -nobak -o /var/tmp/backup-${HOSTNAME}.iso 
/var/tmp/backup
df -h /var/tmp
uptime
# scp -p /var/tmp/backup-${HOSTNAME}.iso 
backup.office.lan:/var/local/backups || :
time rsync ${RSO} /var/tmp/backup-*.iso 
backup.office.lan:/var/local/backups ||:
time rsync ${RSO} /var/tmp/backup-*.iso 192.168.0.1:/var/local/backups/
date

mail:~#

I do the mucking around with tar because mkzftree doesn't do much in the 
way of filtering the source filesystem.

The resultant image is easily read by Linux but on other operating 
systems one needs to run mkzftree to unpack it.




-- 

Cheers
John

-- spambait
1aaaaaaa at computerdatasafe.com.au  Z1aaaaaaa at computerdatasafe.com.au
Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/

do not reply off-list