rsyncing 79gb of data to 250gb drive

Michael Mansour micoots at yahoo.com
Sat May 29 04:20:24 UTC 2004


Hi,

Just to keep things updated here:

 --- Michael Mansour <micoots at yahoo.com> wrote: > Hi,
> 
>  --- Ow Mun Heng <Ow.Mun.Heng at wdc.com> wrote: > On
> Wed, 2004-05-26 at 11:42, David Maynard wrote:
> > > I would suspect that some of the source files
> are
> > "sparse."  Ie. there are
> > > "holes" in the middle of them where space hasn't
> > been allocated in the
> > > source directory.  Some applications that use
> > random file access will create
> > > sparse files.  A simple copy operation that
> > doesn't look for sparse files
> > > will fill in the holes, causing the copy to take
> > up more space than the
> > > original.
> > > 
> > > I haven't tried it (at least not recently), but
> > rsync appears to have a
> > > "--sparse" (-S) option.  Try using that in
> > addition to -av and see if that
> > > changes the results.  GNU tar has a similar
> > option.
> > 
> > Actually, if you're trying to copy all the files
> > from A to B, where B is
> > a Fresh directory, It's advisable to use GNU Tar.
> > it'll speed up the
> > transfer and it'll ensure that permissions are
> kept
> > as they should.
> > 
> > After tha Rsync them for incremental backup etc.
> > 
> > my 2 cents
> 
> For the people following this sage here is the
> result
> of the rsync withOUT the -S option:
> 
> /dev/md6              85419328  79678740   1401448 
> 99% /data01
> /dev/hdf1            241263968 124262976 117000992 
> 52% /data02
> 
> So the problem is sparse files.
> 
> I wonder if tar takes this into account also?
> 
> Michael.

I did another:

rsync -av -S /data01/ /data02

with the following result:

/dev/md6              85419328  79678740   1401448 
99% /data01
/dev/hdf1            241263968  79532584 161731384 
33% /data02

Strange how I got such files to begin with, because on
the original partition it's got 79Gb, I'm assuming
with sparse files, and only when I perform the rsync
(without the -S option) does it grow the sizes on the
rsync.

Not only do I wonder if tar will respond the same way
as rsync without the -S option, but I'm also wondering
if backup software which use rsync to perform backups
would also take this into account...

Michael.


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com





More information about the users mailing list