rsync or dd to clone a hard drive?

[Maxime Alarie]

I have never used dd to clone a hard disk.

I use rsync for a lot of   my backups , and I was wondering what would
be the best tool to clone a  disk.  I don't want 3rd party software
also.  I want built in command  line tools.

I know rsync will resync where it left off if I encounter a problem,
what about dd?  Do I  absolutely have to creat an image before cloning?
Ex: dd if=/dev/sda of=sda.img   or I can use dd if=/dev/sda of=/dev/sdb?

Thanks.

[Jason Brown]

rsync is used primarily if you want to do file level backups.  If you  
wanted to make a complete bare metal restore of a partition or disk,  
then use dd.

It also depends on how much space you have for backups as well, since  
file level backups would be smaller in size then using dd.

On Oct 6, 2010, at 10:58 AM, Maxime Alarie wrote:
> I have never used dd to clone a hard disk.
>
> I use rsync for a lot of   my backups , and I was wondering what  
> would be the best tool to clone a  disk.  I don’t want 3rd party  
> software also.  I want built in command  line tools.
>
> I know rsync will resync where it left off if I encounter a problem,  
> what about dd?  Do I  absolutely have to creat an image before  
> cloning?  Ex: dd if=/dev/sda of=sda.img   or I can use dd if=/dev/ 
> sda of=/dev/sdb?
>
> Thanks.
>
> -- 
> users mailing list
> [e-mail]
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

[JB]

Maxime Alarie <malarie <at> processia.com> writes:
> I have never used dd to clone a hard disk.
>
> I use rsync for a lot of   my backups , and I was wondering
> what would be the best tool to clone a  disk.  I don’t want 3rd
> party software also.  I want built in command  line tools.
>
> I know rsync will resync where it left off if I encounter a
> problem, what about dd?  Do I  absolutely have to creat an image before
> cloning? 
> Ex: dd if=/dev/sda of=sda.img   or I can use dd if=/dev/sda of=/dev/sdb?
>
> Thanks.

Hi,
if you want to clone it, take a look at possible tools as discussed in a thread
here:
22 Aug 07:08	Suvayu Ali  *	Migrating data off a failing drive

JB

[John Aldrich]

On Wed October 6 2010, JB wrote:
> Maxime Alarie <malarie <at> processia.com> writes:
> > I have never used dd to clone a hard disk.
> >
> > I use rsync for a lot of   my backups , and I was wondering
> > what would be the best tool to clone a  disk.  I don’t want 3rd
> > party software also.  I want built in command  line tools.
> >
> > I know rsync will resync where it left off if I encounter a
> > problem, what about dd?  Do I  absolutely have to creat an image
> > before cloning?
> > Ex: dd if=/dev/sda of=sda.img   or I can use dd if=/dev/sda
> > of=/dev/sdb?

Neither. ;-) Use G4L or similar. It's free, it's bootable and it does 
network ->Disk, Disk->Disk or Disk ->Network. :-)

[JD]

On 10/06/2010 07:58 AM, Maxime Alarie wrote:
> I have never used dd to clone a hard disk.
>
> I use rsync for a lot of my backups , and I was wondering what would 
> be the best tool to clone a disk. I don’t want 3^rd party software 
> also. I want built in command line tools.
>
> I know rsync will resync where it left off if I encounter a problem, 
> what about dd? Do I absolutely have to creat an image before cloning? 
> Ex: dd if=/dev/sda of=sda.img or I can use dd if=/dev/sda of=/dev/sdb?
>
> Thanks.

Never run dd on a mounted disk or partition. You will not get a correct
snapshot, and if you restore from such a backup, yo will end up with a
corrupted file system. Why? Because a mounted file system, probably actively
being used by something, will experience changes. These changes will likely
not be reflected in the backup image if the offset where dd is reading is
ahead of where the changes are occurring.

You mentioned clone. I assume you mean you have another drive, partitioned
similarly to what you want to clone, or has a partition that is at least 
equal
in size to the partition you want to clone with dd. Is this correct?
If yes, then both partitions must be unmounted. then running
dd from one partition to the other will not be a problem.
The only issue you MIGHT have is this: if the destination partition is
much larger than the source partition, then you will not see the unused
remainder of the target partition when you mount the target partition.
The filesystem will report "full" if you fill t up-to the size of the 
original
partition. Caveat Emptor!

[Michael D. Setzer II]

On 6 Oct 2010 at 10:58, Maxime Alarie wrote:

Subject:	rsync or dd to clone a hard drive?
Date sent:	Wed, 6 Oct 2010 10:58:50 -0400
From:	"Maxime Alarie" 
To:	
Send reply to:	Community support for Fedora users 
    
> I have never used dd to clone a hard disk.
>
> I use rsync for a lot of my backups , and I was wondering what would be the best tool to 
> clone a disk. I don’t want 3rd party software also. I want built in command line tools.
>
> I know rsync will resync where it left off if I encounter a problem, what about dd? Do I 
> absolutely have to creat an image before cloning? Ex: dd if=/dev/sda of=sda.img or I 
> can use dd if=/dev/sda of=/dev/sdb?

I'm the maintainer of the g4l project that is a disk imaging program. It is 
usually used to boot a machine from the cd with a ramdisk, and then can 
make local or network images using dd and compress. Note: this is a full disk 
or full partition images. Not a file or directory level image.

It also has an option to clone, which is a similar process to you dd option 
going from one disk to another.

ftp://amd64gcc.dyndns.org/g4l-v0.35alpha19.iso
ftp://amd64gcc.dyndns.org/g4l-v0.35a19.devel.tar.gz

or released version on sourceforge.
> Thanks.

+----------------------------------------------------------+
  Michael D. Setzer II -  Computer Science Instructor      
  Guam Community College  Computer Center                  
  mailto:[e-mail]                            
  mailto:[e-mail]
  http://www.guam.net/home/mikes
  Guam - Where America's Day Begins                        
  G4L Disk Imaging Project maintainer 
  http://sourceforge.net/projects/g4l/
+----------------------------------------------------------+

http://setiathome.berkeley.edu (Original)
Number of Seti Units Returned:  19,471
Processing time:  32 years, 290 days, 12 hours, 58 minutes
(Total Hours: 287,489)

BOINC@HOME CREDITS
SETI        10009860.871197   |   EINSTEIN     4660449.360851
ROSETTA      2297938.485565   |   ABC          2771227.136067

[Dan Thurman]

On 10/06/2010 10:06 AM, Michael D. Setzer II wrote:
> On 6 Oct 2010 at 10:58, Maxime Alarie wrote:
>
> Subject:	rsync or dd to clone a hard drive?
> Date sent:	Wed, 6 Oct 2010 10:58:50 -0400
> From:	"Maxime Alarie" 
> To:	
> Send reply to:	Community support for Fedora users 
>    
> > I have never used dd to clone a hard disk.
> >
> > I use rsync for a lot of my backups , and I was wondering what would be the best tool to 
> > clone a disk. I don’t want 3rd party software also. I want built in command line tools.
> >
> > I know rsync will resync where it left off if I encounter a problem, what about dd? Do I 
> > absolutely have to creat an image before cloning? Ex: dd if=/dev/sda of=sda.img or I 
> > can use dd if=/dev/sda of=/dev/sdb?
>
> I'm the maintainer of the g4l project that is a disk imaging program. It is 
> usually used to boot a machine from the cd with a ramdisk, and then can 
> make local or network images using dd and compress. Note: this is a full disk 
> or full partition images. Not a file or directory level image.
>
> It also has an option to clone, which is a similar process to you dd option 
> going from one disk to another.
>
> ftp://amd64gcc.dyndns.org/g4l-v0.35alpha19.iso
> ftp://amd64gcc.dyndns.org/g4l-v0.35a19.devel.tar.gz
>
> or released version on sourceforge.
> > Thanks.
>
> +----------------------------------------------------------+
>   Michael D. Setzer II -  Computer Science Instructor      
>   Guam Community College  Computer Center                  
>   mailto:[e-mail]                            
>   mailto:[e-mail]
>   http://www.guam.net/home/mikes
>   Guam - Where America's Day Begins                        
>   G4L Disk Imaging Project maintainer 
>   http://sourceforge.net/projects/g4l/
> +----------------------------------------------------------+
>
> http://setiathome.berkeley.edu (Original)
> Number of Seti Units Returned:  19,471
> Processing time:  32 years, 290 days, 12 hours, 58 minutes
> (Total Hours: 287,489)
>
> BOINC@HOME CREDITS
> SETI        10009860.871197   |   EINSTEIN     4660449.360851
> ROSETTA      2297938.485565   |   ABC          2771227.136067

Just wondering...

1) How does 4GL compare with Clonezilla?

2) Why aren't "backup/restore/image/clone" programs such as (1) above
    integrated into LiveCDs that have most the Linux core, GUI, Network,
...,
    support?

For the moment, I use rsync for linux copy/clones/moves from
source to target partitions for the same hardware, and it works.

I do these steps:

(1) Boot with LiveCD

(2) # rsync -ahHAX <source> <target>

Notes:
         (a) Optionally add 'z' argument if you want compression and
             add 'v' if you want to see verbosity at the expense of transfer
             speed.
         (b) Abruptly stopped?  No problem, run the above command again.
         (c) <source> and/or <target> can be remote devices but they have
             to be mounted. The <source> ought not be an active running
             OS, can be mounted to /mnt via LiveCD. The hardware for the
             <source> & <target> ought to be exactly the same if one expects
             the <target> OS to be bootable after transfer has
completed.  I have
             done this with (a) remote backup source to target and (b)
between
             two identical laptops transfers via LiveCD on both, and it
works.
         (d) I have not tried to create an rsync "image".  If it were
possible
             to create an rsync "image" with with FULL
acls/permissions/..., how
             would it be possible to perform an rsync "restore"?
             I have not tried this.

(3) Setting MBR on boot or / (boot integrated) partitions:
         # grub
              find /grub/stage1 (or /grub/grub.cfg)
              root (hX,Y)
              setup (hX,Y)
              quit

(4) # touch /.autorelabel; reboot     (for SELinux)

FWIW,
Dan

[Michael D. Setzer II]

On 8 Oct 2010 at 8:52, Daniel B. Thurman wrote:

Date sent:      	Fri, 08 Oct 2010 08:52:27 -0700
From:           	"Daniel B. Thurman" 
To:             	[e-mail]
Subject:        	Re: rsync or dd to clone a hard drive?

Cut --

Good Questions..
> How does 4GL compare with Clonezilla?

The one version seems similar to what g4l does with the cd running, where 
the bigger version has a database system that needs to be setup.

The one big difference is that g4l has a number of kernels on the cd image, 
since some hardware requires the latest kernels, and other hardware works 
with older version. So the user can boot from various kernels to find one that 
works with there hardware. I even have a method that can be used to add it 
to an ntfs partition to boot via grub4dos.

As a side note, it seems at least for downloads the vast number of users are 
using windows for the download? About 200 - 300 per day.
> Why aren't "backup/restore/image/clone" programs such as (1) above
>  integrated into LiveCDs that have most the Linux core, GUI, Network,
>
> ...,
>     support?

I believe both the smaller version of clonezilla and g4l are included on parted 
magic live cd? The g4l script can also be downloaded, and if the support 
programs are there, it can be run from various cds. Have in the past made 
packages that included the script and some programs that distro live cds 
didn't include. Did that for knoppix and finnix, and these same ones worked 
with others.

G4L can backup and restore LVM partitions, but currently can not mount 
them directly. It can use LVM partitions thru the ftp to store images on LVM 
partitions.
> For the moment, I use rsync for linux copy/clones/moves from
> source to target partitions for the same hardware, and it works.
>  I do these steps:
>
>  (1) Boot with LiveCD
>
>  (2) # rsync -ahHAX <source> <target>
>
>      Notes:
>      (a) Optionally add 'z' argument if you want compression and
>          add 'v' if you want to see verbosity at the expense of transfer
>          speed.
>      (b) Abruptly stopped?  No problem, run the above command again.
>      (c) <source> and/or <target> can be remote devices but they have
>          to be mounted. The <source> ought not be an active running
>          OS, can be mounted to /mnt via LiveCD. The hardware for the
>          <source> & <target> ought to be exactly the same if one expects
>          the <target> OS to be bootable after transfer has
>
> completed.  I have
>              done this with (a) remote backup source to target and (b)
> between
>              two identical laptops transfers via LiveCD on both, and it
> works.
>          (d) I have not tried to create an rsync "image".  If it were
> possible
>              to create an rsync "image" with with FULL
> acls/permissions/..., how
>              would it be possible to perform an rsync "restore"?
>              I have not tried this.

The disk and partition images are bit level backups, and so full filesystem is 
saved. Exception is NTFSCLONE backups of ntfspartitions, which are file 
only backups.  Also, since it is a bit level backup, pre-clearing of unused 
space makes the size of images much smaller.
>  (3) Setting MBR on boot or / (boot integrated) partitions:
>      # grub
>           find /grub/stage1 (or /grub/grub.cfg)
>           root (hX,Y)
>           setup (hX,Y)
>           quit

G4L can backup the MBR and partition table separately, or  include it in a full 
disk image.
>  (4) # touch /.autorelabel; reboot     (for SELinux)

Bit level makes all this the same.

Thanks for the questions.

In conclussion, G4L is a tool to do disk and partition images, as contrasted to 
file level backups.
> FWIW,
> Dan
>
> -- 
> users mailing list
> [e-mail]
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

+----------------------------------------------------------+
  Michael D. Setzer II -  Computer Science Instructor      
  Guam Community College  Computer Center                  
  mailto:[e-mail]                            
  mailto:[e-mail]
  http://www.guam.net/home/mikes
  Guam - Where America's Day Begins                        
  G4L Disk Imaging Project maintainer 
  http://sourceforge.net/projects/g4l/
+----------------------------------------------------------+

http://setiathome.berkeley.edu (Original)
Number of Seti Units Returned:  19,471
Processing time:  32 years, 290 days, 12 hours, 58 minutes
(Total Hours: 287,489)

BOINC@HOME CREDITS
SETI        10009860.871197   |   EINSTEIN     4670910.500851
ROSETTA      2304322.602610   |   ABC          2794658.432901

[Andrew Haley]

I've never found anything better than

dump 0f - device | restore rf -

This copies the whole thing, including files with holes, attributes,
and so on.

Andrew.

[Kevin Cummings]

On 10/06/2010 10:58 AM, Maxime Alarie wrote:
> I have never used dd to clone a hard disk.
>
> I use rsync for a lot of   my backups , and I was wondering what would
> be the best tool to clone a  disk.  I don’t want 3^rd party software
> also.  I want built in command  line tools.

I use tar to migrate date from 1 drive to another.  It can copy
permissions and owners. For a short period, it did not copy context, but
now it can.  It will copy directory structures intact and copies links
and symlinks as is (by default).

tar --xattrs --selinux --one-file-system -cf - /dir | (cd /new/path; tar
-xf -)

should do the job as root user.  You can even add the -S option to
handle sparse file efficiently.
> I know rsync will resync where it left off if I encounter a problem,
> what about dd?  Do I  absolutely have to creat an image before cloning? 
> Ex: dd if=/dev/sda of=sda.img   or I can use dd if=/dev/sda of=/dev/sdb?

tar can be told to not overwrite old files (-k), but you probably *want*
to overwrite the incomplete file where it previously died.
> Thanks.