Brain fart: no format option on a pen drive pop-up menu?

[les]

On Sun, 2011-08-28 at 23:10 -0300, Fernando Cassia wrote:
> On Sun, Aug 28, 2011 at 14:30, Tim <ignored_mailbox at yahoo.com.au> wrote:
> >  *Those* people are not the ones to pay attention to
> > when trying to understand the computer.  You are one of those people.
> 
> Oh yes, I'm a complete fool. And ignorant. That's why I started using
> computers at age 8 (a trs-80 model III) and why I learned 6509
> assembler, too. That's why I wrote about 500 articles about hardware
> and software, because I don't know what formatting a removable drive
> is. Gee, I thank the heavens that I ran into you to explain me what
> formatting actually means...
> 
> > You should be thankful that there are nitpickers correcting the mistakes
> > that people make, and think, otherwise Linux would be the complete
> > shambles that Windows is.
> 
> Linux will continue to be the niche market OS if people like you have
> your way. People who come from Windows will want to, gee, _format_ a
> thumb drive either as fat16 or fat32, or ext4 why not, and they will
> find no format command, just mkfs, which, gee, does what they want to
> do. But is an 'alias' included? of course not. It's much better to
> leave them guessing and asking 4000+ times on Linuxquestions "how do I
> bloody format a flash drive"?
> 
> http://www.google.com/#sclient=psy&hl=en&site=&source=hp&q=format+flash+drive+linux+site:linuxquestions.org&btnK=Google+Search&pbx=1&oq=&aq=&aqi=&aql=&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.r_cp.&fp=8611df39e77e29eb&biw=1360&bih=571
> 
> Four thousand results, I kid you not. On Linuxquestions alone.
> 
> And not only that, God save them from entering this list, where some
> will begin an argument about "formatting" not being actually
> formatting, and that formatting does not erase the contents of the
> drive.
> 
> I despise arguments like yours, I really do. And I honestly believe
> people like you prevent Linux from becoming mainstream.
> 
> Hey look, IoMega and HP are language sinners too, they call formatting
> something that, according to you, is not formatting!:
> 
> http://www.iomega.com/support/manuals/zip2a/use_disks.html
> "You can format your Zip disks to quickly erase all of the files on
> the disk, change the format type (Mac or PC), or to repair a disk that
> has developed errors."
> 
> And look, HP too...
> 
> HP USB storage Format Tool
> http://h30499.www3.hp.com/t5/Business-PCs-Deskpro-EVO/Help-with-HP-USB-Disk-Storage-Format-Tool/td-p/1126393
> 
> I guess you'll have to add the whole of the PC industry to your list
> of evil doers.
> Better start writing letters, quick...
> 
> FC
Ok, to a casual user, these statements have meaning, and the origin of
that meaning is historical, and generally related to CP/M.  But the
reality of using a complex operating system and having the requirement
to maintain ones own level of security with the system, means
understanding at a low level what really happens.  In FAT filesystems,
the Partition table tracks segment usage, and tells you what size each
segment is in blocks. A block is a unit of disk usage, and in most FAT
systems is 512 bytes. A segment is some number of blocks, chosen to
reasonably monitor the appropriate disk space.  For FAT16 systems the
numbers chosen limit the total partition size to 2Gbytes.  For FAT 32
the size goes up a lot.  I don't remember the total, and that number is
not germane to the discussion anyway.  Dynamically allocated file
systems utilize INODES to manage the disk space and their space
allocation is more dependent on the available file system, using less
than 1% of the available space to control access to the remaining file
system.

A file entry in the root directory consists of the file name, and a
pointer to the first segment tracker in the partition table.  The
location multiplied by the segment gives the relative location of the
file starting segment.  That segment consists of numerically sequential
blocks of data, which may or may not be physically sequential which is
controlled by the disk setup, and is determined by the access speed of
the electronics compared to the mechanical access time of the data.

Formatting a disk simply redoes the partition table and zeroes the
segment pointers.  It doesn't clean the disk platter.  Deleting the file
means cleaning the segment pointer list for that file and marking the
directory entry as released.  Note that once again the data is NOT
removed.  

When a disk is formatted, a casual user would find no directory entries
listed by the OS, and would assume that the disk is empty.  Ditto for
deleting a file, if the file name disappears (marked unused) and the
segment list is nullified, the disk usage would be reduced in the count
of allocated segments, the file name is no longer reported by the file
system, and to the casual user the file is gone.

Enter a requirement for security, and things are different.  Using
recovery tools, those "deleted files" and "formatted disks" are still
full of data.  And moreover, the file segments contain clues that will
allow the linkages to be recovered.  Thus a formatted disk or a deleted
file can be recovered.  To be secure means to remove all traces of the
file or to completely clean the disk.  With today's disks containing
Terabytes of information, cleaning one can take forever.  It takes
several varieties of writing to the disk to completely obliterate any
trace of the file data, to get the idea, just think of what the disk is
designed to do.  It is designed to hold the magnetic fields for decades.
It will not give up that magnetization easily.  Moreover, the longer the
data was in place, the more embedded it is into the disk coating, at
least until the coating begins to mechanically degrade.

Disk forensics will recover any formatted disks, and can recover files,
even after they have been overwritten a few times.  Understanding this
is vital if you wish to provide security to yourself or your users.  In
most circumstances, the only way to ensure the loss of all data on a
disk is to physically destroy the disk with fire or mechanical
shredding.  

Enter solid state media.  The new flash products rely on physics for
storage.  The data is permanently installed into what you could consider
electrically isolated canisters.  To physically erase that data, a much
greater change in power is required, so the flash systems use a dc to dc
converter to produce a stronger voltage to overcome the storage and
erase the data.  The same method is used to write new data.  But a cell
can only be written to a "one" state, or to a "zero" state, depending on
the design.  Therefore to write a bit into a block, the block must be
temporarily stored, the block erased, the new bit written into the
temporary copy and then that block written to the blank segment.  But
because the erase and write process are physically incrementally
destructive, additional steps are taken to "level the usage" thus the
block you write is typically not physically the same block you erased.
That means that data is left in various places over the solid state
device.  Unfortunately (if you are security conscious), the bits are not
random, they are not totally erased, and they can be recovered using a
different set of forensic tools.  Thus the format, delete and other file
and file system utilities have even less hold on the data than with
rotating disks.

Being familiar with these systems and their internals gives you an edge
in holding your own on personal security and system security.

For example, one early virus would just mark various segments of the
disk as unusable.  This prevents the disc control software from reading
or writing to that segment.  It can allow an intruder to secure a bit of
your disk from your use.  This can be used to hide a virus or to hold
copies of information, for example from a key stroke tracker or mouse
tracker.  That information can later be recovered by the virus and
transmitted clandestinely to a remote host(s).

Understanding the underlying mechanisms allow you to be able to better
understand the strengths and weaknesses of the systems you use.

Feel free to correct me where you see errors, as this was just off the
top of my head.

Regards,
Les H