Brain fart: no format option on a pen drive pop-up menu?

[Rich Mahn]

I think this has more or less been said, but I'll say it this
way.

Regarding the use of the term "format".  The MSDOG format program
actually did (and may still -- i'm not sure) do a "low-level format"
(which is laying down tracks on a disk) on floppy diskettes.  Earlier
microcomputer operating systems like CPM also had programs called
"format" which placed the tracks on media so the heads could find
them and they could be used.

In the '80s Sun Microsystems had an operating system called SunOS, later
named Solaris.  It came with a program called "format".  Last I looked
Solaris still has that program.  Sun's "format" program creates and
modifies the partition table.  That's it.  It does not format the media
in the traditional sense (laying down tracks), nor in the MicroSoft
sense (creating filesystems).  There are actually a few additional
features, but they neither format the media in the original sense, nor
in the Windows sense.  In UNIX type systems the mkfs program is used to
create filesystems.  It is analagous to windows "format" function.

So that's three completely different definitions of the term "format".
Which is why there is so much confusion.  Which of the 3 are you talking
about at the moment?

We might as well blame Microsoft for this problem, even if it was Sun
that started the mis-use of the term.

les <hlhowell at pacbell.net> wrote:

> Formatting a disk simply redoes the partition table and zeroes the
> segment pointers.  It doesn't clean the disk platter.  Deleting the file
> means cleaning the segment pointer list for that file and marking the
> directory entry as released.  Note that once again the data is NOT
> removed.  

Here we are, of course, talking about microsoft's definition of format.

> When a disk is formatted, a casual user would find no directory entries
> listed by the OS, and would assume that the disk is empty.  Ditto for
> deleting a file, if the file name disappears (marked unused) and the
> segment list is nullified, the disk usage would be reduced in the count
> of allocated segments, the file name is no longer reported by the file
> system, and to the casual user the file is gone.
> 
> Enter a requirement for security, and things are different.  Using
> recovery tools, those "deleted files" and "formatted disks" are still
> full of data.  And moreover, the file segments contain clues that will
> allow the linkages to be recovered.  Thus a formatted disk or a deleted
> file can be recovered.  To be secure means to remove all traces of the
> file or to completely clean the disk.  With today's disks containing
> Terabytes of information, cleaning one can take forever.

Agree down to here.

>                                                            It takes
> several varieties of writing to the disk to completely obliterate any
> trace of the file data, to get the idea, just think of what the disk is
> designed to do.  It is designed to hold the magnetic fields for decades.
> It will not give up that magnetization easily.  Moreover, the longer the
> data was in place, the more embedded it is into the disk coating, at
> least until the coating begins to mechanically degrade.

This is just plain wrong.  For modern hard drives (manufactured after
1994), it is sufficient to overwrite the disk once, with any pattern you
desire.  I'm not talking about floppy diskettes or core memory here, I'm
talking about hard disks.
Also, I would bet that the longer the data had not been re-written, the
less embeeded it is, not more embedded.  Again, we're not talking about
core memory.
Google "Advisory No. LAA-006-2004" for NSA's statement on this.

> Disk forensics will recover any formatted disks, and can recover files,
> even after they have been overwritten a few times.  Understanding this
> is vital if you wish to provide security to yourself or your users.  In
> most circumstances, the only way to ensure the loss of all data on a
> disk is to physically destroy the disk with fire or mechanical
> shredding.  

Untrue wrt overwriting--at least for hard drives manufactured after
1994.  Possibly true for floppy diskettes, especially the 5" or 8" ones
and some of the old hard disks, like the RM03s.  But for modern hard
drives, a single overwrite pass makes it impossible to recover prior data
from that particular location.

That's not to say there aren't other ways to recover it--but not from
the disk location that was overwritten.  For example, some RAID levels
can recover full disks of data after the removal and destruction of the
disk.  There are backup systems.  Lots of other ways to recover data,
but NOT from the locations that are overwritten.

> Enter solid state media.  The new flash products rely on physics for
> storage.  The data is permanently installed into what you could consider
> electrically isolated canisters.  To physically erase that data, a much
> greater change in power is required, so the flash systems use a dc to dc
> converter to produce a stronger voltage to overcome the storage and
> erase the data.  The same method is used to write new data.  But a cell
> can only be written to a "one" state, or to a "zero" state, depending on
> the design.  Therefore to write a bit into a block, the block must be
> temporarily stored, the block erased, the new bit written into the
> temporary copy and then that block written to the blank segment.  But
> because the erase and write process are physically incrementally
> destructive, additional steps are taken to "level the usage" thus the
> block you write is typically not physically the same block you erased.
> That means that data is left in various places over the solid state
> device.  Unfortunately (if you are security conscious), the bits are not
> random, they are not totally erased, and they can be recovered using a
> different set of forensic tools.  Thus the format, delete and other file
> and file system utilities have even less hold on the data than with
> rotating disks.


2My comments above are not intended to apply to solid state media.  Only
hard disk drives.  (just to clarify)

You didn't mention CDRWs or DVDRWs.  Your comments on solid state media
more or less apply to them as well.

> Being familiar with these systems and their internals gives you an edge
> in holding your own on personal security and system security.

completely agree


> Understanding the underlying mechanisms allow you to be able to better
> understand the strengths and weaknesses of the systems you use.

didn't I just agree to this. ;)


> Feel free to correct me where you see errors, as this was just off the
> top of my head.

I did.