No

[Les]

On Sun, 2007-09-02 at 21:23 +0100, Ian Malone wrote:
> Karl Larsen wrote:
> 
> >    I will wear it and just read info dd for about the 4th time in 5 
> > days. The first time I read it I was not aware that dd sends Everything 
> > including the file system. The info dd does not say this.
> > 
> 
> "Copy a file, converting and formatting according to the operands."
> You applied it to a block device.  Maybe it would have been worth
> knowing what you were doing.
> 
> >    I tried to copy this f7 which is about 8 Gbytes to another partition 
> > that was 10 Gbytes. The transfer failed because the 8 Gbyte f7 was from 
> > a 40 Gbyte partition :-)
> > If you read everything in info dd you will not see this listed.
> > 
> 
> 10 < 40.  Not sure what more needs to be said here.
> 
> >    So it may be time to get the author of the dd info to add something 
> > about large copies and the use of the Rescue CD.
> > 
> 
> You're serious?
> 
> -- 
> imalone
> 
Hi, Ian,
	This is one of the aspects of computers that someone with experience
understands and someone else may not.  

Karl,
	The disk contains not only the partition table, files and data, but
also many blocks of data that contain information about how the files
are stored, the superblocks.  These are linked.  If one changes, they
all change.  The dd utility looks at the disk and finds its size, then
copys that to the output file system.  This will include the superblocks
and everything else that is tied to the disk structure that is visible
to the outside world.  There is also data on the disk that is not
visible to the outside world that is used by the interface, but dd won't
see that, and won't copy it (doesn't have access).  The disk structure
then is spread out over the full size of the disk.  Moreover, when you
edit or update a file, it is not generally done "in place", but rather a
new file is created, then the old one is erased.  This means a disk can
have voids in its cylinders (not a bad thing, because this process
prevents excessive writes to the same areas of the disk to some extent,
thus prolonging the life of the disk slightly).

	As a result, the "clone tools" such as dd and a few others that I used
to use, will copy the disk literally.  As long as the disks are the same
basic physical structure this works.  But if the size is different, the
cylinder size is different, or the sector size is different then it will
not work well, although it may complete successfully.  If the cylinders
have more sectors, some will be left blank, and due to the design of the
superblocks, not accessible after the clone is complete.  If the
cylinders have fewer sectors (not a likely scenario today, but used to
happen), then the clone would overwrite on successive sectors past the
physical number unless it detected the error.  If the disk is faster or
slower in rotation, it can also cause problems, as can relative
bandwidth of the heads, but generally the controller will attempt to set
this correctly.  Thus dd has evolved over the years to detect most of
the possible errors and prompt you to not continue.  However, since dd
is also a catastrophe tool, it will proceed if told to do so, so that
someone in a bad situation may be able to clone the disk to a larger
disk and hopefully recover their data.  It has been pointed out in this
thread that this is also sometimes the process used by computer
forensics folks to clone the disk to avoid tampering with its contents,
and then work only on the clone.

  This is one of those tools that can do a lot of damage if used
unwisely, and so as many folks here told you, using the same identical
hardware is critical to ensuring success in the most basic manner.  

Another issue that arises in dynamic systems that are interrupt and
demand driven like all modern multithreaded systems is the possibility
of file modifications.  This would result in a "snap shot" of a file
system in transition, which could possibly cause catastrophic failure
should the cloned system be used to boot.  One of the responders I felt
gave a really good example of a book being written as it was copied.

	And, no, the man and info pages do not detail exactly what dd does or
the potential interactions.  However I do believe that they say to not
perform the operation on mounted systems, and that the systems should be
the same or at least the same size.  

	In short, man pages are abbreviated to "just the facts, Ma'am" as Joe
Friday used to say.  Therefore using commands just based upon man one
should always follow very carefully every argument, every precaution,
and not do anything else.  

	Otherwise, you need more information.  This is true of most every *nix
command, as most have evolved over the years to much greater capability
than was originally envisioned.  The original mantra was for each
command to do one thing, but do it well and be capable of chaining to
other command through pipes or I/O redirection to accomplish complex
tasks.  However as systems have grown, our ability to segment the
complex tasks is being overwhelmed, so commands get more and more
capabilities, necessitating manuals for some of the commands, and more
understanding of the OS and system functions and processes. The command
dd falls into this category, but so do most other commands.

I do not believe that man should be your only source, but I do agree
that one should read and follow the man recommendations religiously.
This applies to old hands as well as new.  I have seen "we've always
done it this way" go too far wrong in many cases.  Be careful.

	And by the way, Karl, you did the right thing.  Write down what you
did.  Get it clarified, and improve it, whether for yourself or for
wider dissemination.  That is the true spirit of open source.  The
gentleman here said make a contribution to man.  I believe you have made
a contribution, perhaps not to man, but to several folks out there who
did not understand what was happening or why.  I hope I have helped
here.  But if not... at least I tried.  Good luck to you both.

Regards,
Les H