optimesing NFS Server for e.g. remote Backup
John Summerfied
debian at herakles.homelinux.org
Thu Feb 16 14:25:59 UTC 2006
Tony Molloy wrote:
> On Thursday 16 February 2006 07:45, Götz Reinicke wrote:
>
>>Hi,
>>
>>I have some RHEL4 servers and want to set up a central backup system
>>with a "big" harddisk. I found a very nice shell-script, doing
>>incremental backups using tar.
>>
>
>
>>Any ideas what coul be wrong? The Backups are about 10 GB +- up to 100
>>GB in the future.
>
>
> With that amount of data to backup don't use tar, use rsync instead. Rsync
> is much faster as it only copies the parts of files which have changed.
> An excellent backup system using rsync to try is BackupPC
> < backuppc.sourceforge.net >. It does full and incremental backups using
> rsync.
Some time ago, I tried backing up our server over the Internet using
rsync, and even after discussion with the developers and updating to the
latest version, it was way too slow.
It took enormous amounts of RAM and swap (fortunately, depite making
swap several times RAM it didn't induce thrashing), and never completed
in a reasonable time.
Now, I have a devious script that creates a compressed ISO using
mkzftree and mkisofs and then uses rsync to update a remote copy over
ADSL. The longest time to update the remote copy is a whisker over the
hour which we think entirely reasonable.
It takes around five minutes to copy over the local LAN.
Here is the script: watch for line-wrap.
mail:~# cat /root/bin/bu.image
#!/bin/bash -e
RSO="--recursive --links --hard-links --perms --owner --group --devices
--times"
RSO="${RSO} --sparse --one-file-system --rsh=/usr/bin/ssh --delete "
RSO="${RSO} --delete-excluded --relative --stats "
RSO="${RSO} --numeric-ids --timeout=7200 ${EXC}"
trap '/etc/init.d/postfix start' EXIT
/etc/init.d/postfix stop
date
[ -d /mnt/backup ] || mkdir /mnt/backup
Image=/var/tmp/backup.img
umount /mnt/backup || :
rm -fr ${Image} /var/tmp/backup
dd if=/dev/zero of=${Image} count=0 bs=1024 \
seek=$((5*1024*1024))
ls -shog ${Image}
mke2fs -Fq ${Image}
mount -o loop ${Image} /mnt/backup/
sfdisk >/etc/disktab -d /dev/hda
tar clC / --exclude=backup.img --exclude=/tmp --exclude=/mnt\/*
--exclude=/var/lock --exclude=swapfil\* \
--exclude=/var/autofs --exclude=lost+found --exclude=/var/tmp
--exclude=/var/local --exclude=squid-cache \
--exclude=/var/spool/cyrus/mail-backup \
/ /boot /home /var \
| buffer -m $((2*1024*1024)) -p 75\
| tar xpC /mnt/backup || { df -h ; exit ; }
ls -go /mnt/backup
rm ${Image}
/etc/init.d/postfix start && trap -
mkzftree --one-filesystem /mnt/backup/ /var/tmp/backup
uptime
df -h
umount /mnt/backup/
mkisofs -R -z -quiet -nobak -o /var/tmp/backup-${HOSTNAME}.iso
/var/tmp/backup
df -h /var/tmp
uptime
# scp -p /var/tmp/backup-${HOSTNAME}.iso
backup.office.lan:/var/local/backups || :
time rsync ${RSO} /var/tmp/backup-*.iso
backup.office.lan:/var/local/backups ||:
time rsync ${RSO} /var/tmp/backup-*.iso 192.168.0.1:/var/local/backups/
date
mail:~#
I do the mucking around with tar because mkzftree doesn't do much in the
way of filtering the source filesystem.
The resultant image is easily read by Linux but on other operating
systems one needs to run mkzftree to unpack it.
--
Cheers
John
-- spambait
1aaaaaaa at computerdatasafe.com.au Z1aaaaaaa at computerdatasafe.com.au
Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/
do not reply off-list
More information about the users
mailing list