advice for data recovery

[Dave Stevens]

Hello All,

I am trying to recover .jpgs that were on a SATA drive that was formatted by 
mistake. No backup of course. Foremost has done a wonderful job of recovering 
several tens of thousands of files. Unfortunately many of them are either 
irrelevant (cached web fragments, etc.) or damaged. The most common type of 
damage is shown when trying to view them in Nautilus, when I get a message 
saying, "unsupported marker type."

It seems about 30% - 40% of the files recovered are damaged in this way.

They aren't my images and I am not able to gauge what is worthwhile or not but 
I would like to do some triage by only considering those of a certain minimum 
size (easy to do) and not damaged (no idea.)

So does anyone know of a program I can use to only copy files that are not 
damaged? I can sort out the teenies, but don't see how to proceed after that.

TIA.

Dave

[Waleed Harbi]

Try Scalpel
http://www.digitalforensicssolutions.com/Scalpel/
or Foremost
http://foremost.sourceforge.net/

On Tue, Nov 11, 2008 at 12:56 AM, Dave Stevens  wrote:
> Hello All,
>
> I am trying to recover .jpgs that were on a SATA drive that was formatted
> by
> mistake. No backup of course. Foremost has done a wonderful job of
> recovering
> several tens of thousands of files. Unfortunately many of them are either
> irrelevant (cached web fragments, etc.) or damaged. The most common type of
> damage is shown when trying to view them in Nautilus, when I get a message
> saying, "unsupported marker type."
>
> It seems about 30% - 40% of the files recovered are damaged in this way.
>
> They aren't my images and I am not able to gauge what is worthwhile or not
> but
> I would like to do some triage by only considering those of a certain
> minimum
> size (easy to do) and not damaged (no idea.)
>
> So does anyone know of a program I can use to only copy files that are not
> damaged? I can sort out the teenies, but don't see how to proceed after
> that.
>
> TIA.
>
> Dave
>
> --
> Canada must refuse to be entangled in any more wars fought to make the
> world
> safe for capitalism.
>
> -- The Regina Manifesto, 1933
>
> --
> fedora-list mailing list
> [e-mail]
> To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
> Guidelines:
> http://fedoraproject.org/wiki/Communicate/MailingListGuidelines

[Robin Laing]

Dave Stevens wrote:
> Hello All,
>
> I am trying to recover .jpgs that were on a SATA drive that was formatted by 
> mistake. No backup of course. Foremost has done a wonderful job of recovering 
> several tens of thousands of files. Unfortunately many of them are either 
> irrelevant (cached web fragments, etc.) or damaged. The most common type of 
> damage is shown when trying to view them in Nautilus, when I get a message 
> saying, "unsupported marker type."
>
> It seems about 30% - 40% of the files recovered are damaged in this way.
>
> They aren't my images and I am not able to gauge what is worthwhile or not but 
> I would like to do some triage by only considering those of a certain minimum 
> size (easy to do) and not damaged (no idea.)
>
> So does anyone know of a program I can use to only copy files that are not 
> damaged? I can sort out the teenies, but don't see how to proceed after that.
>
> TIA.
>
> Dave

My sympathies.  I did the same thing.  I did a backup but forgot one sub 
folder.  Shame on me.

In the recovery, I ended up with 180Gigs of recovered files to go 
through.  Not pleasant.

I have not found a program that can check and confirm the accuracy of 
the recovered files as they are all listed as images if I type in
    file {file name}

What I have done is use "GQview" and look at the thumbnails.

My procedure is/was this.

With all the recovered images, I wrote a short bash script to sort them 
into smaller sub-directories.  This cut the number of images down to 10K 
or 20K per directory.

Next I opened the directory in GQview.  With thumbnails turned on, this 
takes minutes, depending on the size of the folder.

I then sort the directory by file size.  This helps to eliminate smaller 
files from having to be viewed.  I guess I could write a script to rm 
these files anyway.  See later.

Now with the thumbnails, I can quickly scan the directory for images.  I 
will use the mouse to drag and drop the file in the parent directory. 
When scanning, the scrolling slows/stops for real images over corrupted 
files.

I know my images from my camera are larger than 2M each but the recovery 
software would pull any images it can and thus I sometimes have 
recovered a smaller version of the image where the full image is gone.

I still have over 100Gig to go through.

I now backup my photos to DVD as soon as I take them off the stick.

One note about recovery software.  As I did this at home, I cannot 
remember what program I used.  But what I did find out was it didn't 
look at the headers for the images from my camera.  I needed to change 
the header info in a configuration file.  I did a test on a memory stick 
to ensure I was looking at my images.

[Todd Denniston]

Dave Stevens wrote, On 11/10/2008 04:56 PM:
> Hello All,
>
> I am trying to recover .jpgs that were on a SATA drive that was formatted by 
> mistake. No backup of course. Foremost has done a wonderful job of recovering 
> several tens of thousands of files. Unfortunately many of them are either 
> irrelevant (cached web fragments, etc.) or damaged. The most common type of 
> damage is shown when trying to view them in Nautilus, when I get a message 
> saying, "unsupported marker type."
>
> It seems about 30% - 40% of the files recovered are damaged in this way.

Assuming that all the data is really there but you have extra on the end, then 
you might be able to use convert to read the input and only output picture data.

identify might be able to help you sort images from not images, and corruption.
> They aren't my images and I am not able to gauge what is worthwhile or not but 
> I would like to do some triage by only considering those of a certain minimum 
> size (easy to do) and not damaged (no idea.)
>
> So does anyone know of a program I can use to only copy files that are not 
> damaged? I can sort out the teenies, but don't see how to proceed after that.

man ImageMagick
man identify
man convert

[Mikkel]

Todd Denniston wrote:
> Dave Stevens wrote, On 11/10/2008 04:56 PM:
> > Hello All,
> >
> > I am trying to recover .jpgs that were on a SATA drive that was
> > formatted by mistake. No backup of course. Foremost has done a
> > wonderful job of recovering several tens of thousands of files.
> > Unfortunately many of them are either irrelevant (cached web
> > fragments, etc.) or damaged. The most common type of damage is shown
> > when trying to view them in Nautilus, when I get a message saying,
> > "unsupported marker type."
> >
> > It seems about 30% - 40% of the files recovered are damaged in this way.
>
> Assuming that all the data is really there but you have extra on the
> end, then you might be able to use convert to read the input and only
> output picture data.
>
> identify might be able to help you sort images from not images, and
> corruption.
> > They aren't my images and I am not able to gauge what is worthwhile or
> > not but I would like to do some triage by only considering those of a
> > certain minimum size (easy to do) and not damaged (no idea.)
> >
> > So does anyone know of a program I can use to only copy files that are
> > not damaged? I can sort out the teenies, but don't see how to proceed
> > after that.
>
> man ImageMagick
> man identify
> man convert

I wounder if Jpeg Rescue would help in this case?

http://codesink.org/recover.html

Mikkel

[Robin Laing]

Mikkel L. Ellertson wrote:
> Todd Denniston wrote:
> > Dave Stevens wrote, On 11/10/2008 04:56 PM:
> > > Hello All,
> > > So does anyone know of a program I can use to only copy files that are
> > > not damaged? I can sort out the teenies, but don't see how to proceed
> > > after that.
> >
> > man ImageMagick
> > man identify
> > man convert
>
> I wounder if Jpeg Rescue would help in this case?
>
> http://codesink.org/recover.html
>
> Mikkel

I have not tried identify or convert and will have to test them.

I use ImageMagick and tried many things there but none of them worked.

I did use Jpeg Rescue but it didn't work in my case.  It missed many 
images.  Again, it could be a header issue between different Jpeg files. 
  I found that my camera's header was different than the one in one 
particular program.  I think it was foremost.  This was confirmed with a 
test on a memory card.

All images that were recovered in my case will say that they are valid 
jpg images if I just use 'file {image.jpg}'

[Robin Laing]

Robin Laing wrote:
> Mikkel L. Ellertson wrote:
> > Todd Denniston wrote:
> > > Dave Stevens wrote, On 11/10/2008 04:56 PM:
> > > > Hello All,
> > > > So does anyone know of a program I can use to only copy files that are
> > > > not damaged? I can sort out the teenies, but don't see how to proceed
> > > > after that.
> > >
> > > man ImageMagick
> > > man identify
> > > man convert
> >
> > I wounder if Jpeg Rescue would help in this case?
> >
> > http://codesink.org/recover.html
> >
> > Mikkel
>
> I have not tried identify or convert and will have to test them.

I played with identify and convert (which to my embarrassment are part 
of ImageMagick) and found identify did provide necessary info.  Now 
working on a script to automate the process.

Of course, identify will list all damaged files as damaged and yet they 
may be partially usable.

More playing.