Hi everyone,
I make several backup with bzip2, it seems that he has the better compression.
I do my backup like this for several folders.
tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
When i test the archive with bzip2 -t archive.tar.bz2, everything is correct when the archive is under 1 Go. But i get CRC error when it's upper ....
Has anyone experience this ?
-- Franck
On Friday 30 December 2005 23:56, Franck Y wrote:
Hi everyone,
I make several backup with bzip2, it seems that he has the better compression.
I do my backup like this for several folders.
tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
When i test the archive with bzip2 -t archive.tar.bz2, everything is correct when the archive is under 1 Go. But i get CRC error when it's upper ....
Has anyone experience this ?
No, other than I've also heard that bzip2 has problems with source streams above 2GB, which would explain the effect your are seeing.
As an amanda user myself, it seems rather foolish to make a single backup file that big considering what you have lost if its unrecoverable, and bz2 stuff can be entirely too sensitive to the medium its on. Its demonstrated that to me on many occasions.
"gzip best", OTOH, is considerably more robust in my experience, and faster in the compression cycle than bz2. And, when you have a good backup manager, the individual files can be made fairly smallish so that each file in the archive remains a manageable size. My amanda disklist has 44 entries scattered over 2 machines because I've purposely made each entry only one, under a gigabyte of raw data, directory tree. Having smallish files means amanda can more easily manage the storage to make maximum use of the tape it is allowed to use each night, and I've often filled the tape to 98% capacity without ever hitting an EOT from the drive. With a mixture of entries that compress, and those that aren't (because they already are, like rpms & tar.gz's etc) gzip can routinely reduce 16GB of raw data into 6GB on the tape, so I don't consider the better compression performance of bz2 to be an advantage other than in the eye of the beholder. Storage today is cheap, use what works more dependably rather than that which may work slightly better, but only 99% of the time. For backups, a 1% failure rate is unacceptable to me.
-- Franck
Franck Y wrote:
Hi everyone,
I make several backup with bzip2, it seems that he has the better compression.
I do my backup like this for several folders.
tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
When i test the archive with bzip2 -t archive.tar.bz2, everything is correct when the archive is under 1 Go. But i get CRC error when it's upper ....
Has anyone experience this ?
I have SUSE 10 on my laptop where I've been playing with some fairly big files. I just tested a smaller one, a little over 6 Gbytes compressed,
It's fine.
My files are not tarballs, they're images of paritions and disks, and I did not pipe them into bzip2, I wrote the data to the files and then compressed them. I don't know whether the different way I compressed my data is relevant, but it might be.
I don't use tar that way, I always tell tar to do the compression. It should not be different, but it might be.
Gene, I ll try tonight the gzip contribution, for all my file. (mostly Office documents).
I have a big forder wich is 11 Go without any compression. With bz2 it compress to 4.1 Go which is nice because it fit in a DVD With gzip it take 4.8Go, and i have to take another dvd or Cd to burn it...
I cannot tolerate neither a failure rate even if it's only 0.01 %. All of this folder are extermely important.
Ig gzip is more robust, i do not have another choice...
Franck
On 12/31/05, Gene Heskett gene.heskett@verizon.net wrote:
On Friday 30 December 2005 23:56, Franck Y wrote:
Hi everyone,
I make several backup with bzip2, it seems that he has the better compression.
I do my backup like this for several folders.
tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
When i test the archive with bzip2 -t archive.tar.bz2, everything is correct when the archive is under 1 Go. But i get CRC error when it's upper ....
Has anyone experience this ?
No, other than I've also heard that bzip2 has problems with source streams above 2GB, which would explain the effect your are seeing.
As an amanda user myself, it seems rather foolish to make a single backup file that big considering what you have lost if its unrecoverable, and bz2 stuff can be entirely too sensitive to the medium its on. Its demonstrated that to me on many occasions.
"gzip best", OTOH, is considerably more robust in my experience, and faster in the compression cycle than bz2. And, when you have a good backup manager, the individual files can be made fairly smallish so that each file in the archive remains a manageable size. My amanda disklist has 44 entries scattered over 2 machines because I've purposely made each entry only one, under a gigabyte of raw data, directory tree. Having smallish files means amanda can more easily manage the storage to make maximum use of the tape it is allowed to use each night, and I've often filled the tape to 98% capacity without ever hitting an EOT from the drive. With a mixture of entries that compress, and those that aren't (because they already are, like rpms & tar.gz's etc) gzip can routinely reduce 16GB of raw data into 6GB on the tape, so I don't consider the better compression performance of bz2 to be an advantage other than in the eye of the beholder. Storage today is cheap, use what works more dependably rather than that which may work slightly better, but only 99% of the time. For backups, a 1% failure rate is unacceptable to me.
-- Franck
-- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
-- Franck
John, I'll try with recompiling bzip2... Hope it will work... Happy new year! Franck
On 12/31/05, John Summerfied debian@herakles.homelinux.org wrote:
Franck Y wrote:
Hi everyone,
I make several backup with bzip2, it seems that he has the better compression.
I do my backup like this for several folders.
tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
When i test the archive with bzip2 -t archive.tar.bz2, everything is correct when the archive is under 1 Go. But i get CRC error when it's upper ....
Has anyone experience this ?
I have SUSE 10 on my laptop where I've been playing with some fairly big files. I just tested a smaller one, a little over 6 Gbytes compressed,
It's fine.
My files are not tarballs, they're images of paritions and disks, and I did not pipe them into bzip2, I wrote the data to the files and then compressed them. I don't know whether the different way I compressed my data is relevant, but it might be.
I don't use tar that way, I always tell tar to do the compression. It should not be different, but it might be.
--
Cheers John
-- spambait 1aaaaaaa@computerdatasafe.com.au Z1aaaaaaa@computerdatasafe.com.au Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/
do not reply off-list
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
-- Franck
-
Franck Y -
Gene Heskett -
John Summerfield