input/output error on disk(?)

[Charles Zeitler]

when trying to copy a certain file, i get the message:
cp: reading 'file' :  Input/output error

i took the volume it was on offline, & did a forced
e2fsck.  same thing.

smartctl -H tells me the drive 'passes' .

i can change the name with mv, no problem there.

are there any other (non-destructive) steps i can take?

thanks,

charles zeitler

--

Do What Thou Wilt
       Shall  Be
     The Whole  of
        The Law

-Aleister Crowley

[Sam Varshavchik]

charles zeitler writes:
> when trying to copy a certain file, i get the message:
> cp: reading 'file' :  Input/output error
>
> i took the volume it was on offline, & did a forced
> e2fsck.  same thing.
>
> smartctl -H tells me the drive 'passes' .
>
> i can change the name with mv, no problem there.
>
> are there any other (non-destructive) steps i can take?

Yes. Get a new hard drive, and copy all of your files there, while you still 
have the time to do it.

[Ambrogio]

Il giorno gio, 15/10/2009 alle 00.29 -0500, charles zeitler ha scritto:
> when trying to copy a certain file, i get the message:
> cp: reading 'file' :  Input/output error

How many files you can't copy?
> i took the volume it was on offline, & did a forced
> e2fsck.  same thing.

Did you made the filesystem check with the surface scan?

What is the error logged on /var/log/messages?

Bye
 Ambrogio

[Tony Nelson]

On 09-10-15 01:29:22, charles zeitler wrote:
> when trying to copy a certain file, i get the message:
> cp: reading 'file' :  Input/output error
>
> i took the volume it was on offline, & did a forced
> e2fsck.  same thing.
>
> smartctl -H tells me the drive 'passes' .
>
> i can change the name with mv, no problem there.
>
> are there any other (non-destructive) steps i can take?

It is likely that the disk has one or more bad blocks.  Use
`smartctl -a /dev/sdx` to see all the data (or -A to see only the 
attributes) and look at the raw values of Current_Pending_Sector and 
Reallocated_Event_Count.  The first reflects unfixed problems, and the 
second fixed problems.  If both numbers are low, I would just keep 
using the drive, though I would first do a long test with `smartctl -t 
long /dev/sdx` (wait until the indicated time and then do a `smartctl -
a /dev/sdx` to see what happened) and then enabled Automatic Offline 
Testing with `smartctl -o on /dev/sdx` (this will help keep up with 
sectors as they go bad, without unnecessary data loss).

A program such as ddrescue (`yum install ddrescue`, `info ddrescue`) 
may help to salvage the damaged file(s).

[Charles Zeitler]

thanks for the info.

On Thu, Oct 15, 2009 at 12:30 PM, Tony Nelson
 wrote:
> On 09-10-15 01:29:22, charles zeitler wrote:
> > when trying to copy a certain file, i get the message:
> > cp: reading 'file' :  Input/output error
>
> It is likely that the disk has one or more bad blocks.  Use
> `smartctl -a /dev/sdx` to see all the data (or -A to see only the
> attributes) and look at the raw values of Current_Pending_Sector and
> Reallocated_Event_Count.  The first reflects unfixed problems, and the
> second fixed problems.  If both numbers are low,

13 & 0

I would just keep
> using the drive, though I would first do a long test with `smartctl -t
> long /dev/sdx` (wait until the indicated time and then do a `smartctl -
> a /dev/sdx` to see what happened)

should have more data from this in 4 hours.

and then enabled Automatic Offline
> Testing with `smartctl -o on /dev/sdx` (this will help keep up with
> sectors as they go bad, without unnecessary data loss).
>
> A program such as ddrescue (`yum install ddrescue`, `info ddrescue`)
> may help to salvage the damaged file(s).
>
> --

the file itself is not important, but i don't want the error
to migrate to one that is..

charles zeitler

[Tony Nelson]

On 09-10-15 16:28:03, charles zeitler wrote:
> thanks for the info.
>
> On Thu, Oct 15, 2009 at 12:30 PM, Tony Nelson
>  wrote:
> > On 09-10-15 01:29:22, charles zeitler wrote:
> > > when trying to copy a certain file, i get the message:
> > > cp: reading 'file' :  Input/output error
> >
> > It is likely that the disk has one or more bad blocks.  Use
> > `smartctl -a /dev/sdx` to see all the data (or -A to see only the
> > attributes) and look at the raw values of Current_Pending_Sector 
> > and Reallocated_Event_Count.  The first reflects unfixed problems, 
> > and the second fixed problems.  If both numbers are low,
>
> 13 & 0

13 is a bit high considering that it reflects only the bad blocks that 
have been noticed, but it could still be from a single bad area.  The 
results from the long test will help decide.
> > I would just keep using the drive, though I would first do a long 
> > test with `smartctl -t long /dev/sdx` (wait until the indicated 
> > time and then do a `smartctl - a /dev/sdx` to see what happened)
>
> should have more data from this in 4 hours.

OK.
> > and then enabled Automatic Offline
> > Testing with `smartctl -o on /dev/sdx` (this will help keep up with
> > sectors as they go bad, without unnecessary data loss).
> >
> > A program such as ddrescue (`yum install ddrescue`, `info 
> > ddrescue`) may help to salvage the damaged file(s).
>
> the file itself is not important, but i don't want the error
> to migrate to one that is..

That won't happen, as the drive will use a good block to replace the 
bad block when it is written to.  The real question is how many more 
blocks will go bad, and the long test should give a hint about that.

[Charles Zeitler]

On Thu, Oct 15, 2009 at 4:51 PM, Tony Nelson
 wrote:
> On 09-10-15 16:28:03, charles zeitler wrote:
> > thanks for the info.

well, the results are in.

the  "Self-test execution status" says that
"the read element of the test failed".

later, it shows:
Num  Test_Description    Status                  Remaining
# 1  Extended offline    Completed: read failure       90%

LifeTime(hours)  LBA_of_first_error
    7091         686876074

Current_Pending_Sector & Reallocated_Event_Count
are unchanged. ( but Offline_Uncorrectable reads at 12 ).

does this mean i have 90% unchecked? and 12 uncorrectable blocks?

charles zeitler

Do What Thou Wilt
        Shall  Be
      The Whole  of
         The Law

-Aleister Crowley

[Tony Nelson]

On 09-10-15 21:27:50, charles zeitler wrote:
> On Thu, Oct 15, 2009 at 4:51 PM, Tony Nelson
>  wrote:
> > On 09-10-15 16:28:03, charles zeitler wrote:
> > > thanks for the info.
>
> well, the results are in.
>
> the  "Self-test execution status" says that
> "the read element of the test failed".
>
> later, it shows:
> Num  Test_Description    Status                  Remaining
> # 1  Extended offline    Completed: read failure       90%
>
> LifeTime(hours)  LBA_of_first_error
>     7091         686876074
>
> Current_Pending_Sector & Reallocated_Event_Count
> are unchanged. ( but Offline_Uncorrectable reads at 12 ).
>
> does this mean i have 90% unchecked? and 12 uncorrectable blocks?

Yes, it seems to have given up at 12 blocks.  The trick is to find what 
files those are and deal with them, so that the test can be run again 
to see what else it might find.  Once the damaged files are found, you 
can decide whether to recover them, restore them from backup or some 
other source, or just delete them.

I don't know of anything specifically intended to find the damaged 
files.  e2fsck will map out bad blocks, but doesn't (AFAIK) tell one 
which files are damaged.  I think tar can be used to find such files, 
but I'm not sure.  As you know of one file that has a problem, I 
suggest trying this command on the directory which contains that file:

# tar -cf - --ignore-failed-read /path/to/bad/file's/dir >/dev/null

Possibly -v will also be needed.  In that case, the full scan should 
probably write the messages to a file:

# tar -cvf - --ignore-failed-read --one-file-system / >/dev/null \
      2>/some/other/volume/tarfiles.txt

I don't happen to have any bad blocks to try this on.

[Charles Zeitler]

thanks for the help...

On Thu, Oct 15, 2009 at 9:14 PM, Tony Nelson
> > well, the results are in.
> >
> > does this mean i have 90% unchecked? and 12 uncorrectable blocks?
>
> Yes, it seems to have given up at 12 blocks.  The trick is to find what
> files those are and deal with them, so that the test can be run again
> to see what else it might find.  Once the damaged files are found, you
> can decide whether to recover them, restore them from backup or some
> other source, or just delete them.
>
> I don't know of anything specifically intended to find the damaged
> files.  e2fsck will map out bad blocks, but doesn't (AFAIK) tell one
> which files are damaged.  I think tar can be used to find such files,
> but I'm not sure.  As you know of one file that has a problem, I
> suggest trying this command on the directory which contains that file:
>
> # tar -cf - --ignore-failed-read /path/to/bad/file's/dir >/dev/null
>
> Possibly -v will also be needed.  In that case, the full scan should
> probably write the messages to a file:
>
> # tar -cvf - --ignore-failed-read --one-file-system / >/dev/null \
>      2>/some/other/volume/tarfiles.txt
>
> I don't happen to have any bad blocks to try this on.
>
> --

tar seems not to read files that are headed to /dev/null (?)
trying cat instead. since i am working with hundreds of
gigabytes... it could take awhile.

charles zeitler

[Tony Nelson]

On 09-10-16 00:00:16, charles zeitler wrote:
> thanks for the help...
>
> On Thu, Oct 15, 2009 at 9:14 PM, Tony Nelson
> > I don't know of anything specifically intended to find the damaged
> > files.  e2fsck will map out bad blocks, but doesn't (AFAIK) tell 
> > one which files are damaged.  I think tar can be used to find such,
> > files but I'm not sure.  As you know of one file that has a 
> > problem, I suggest trying this command on the directory which 
> > contains that file:
> >
> > # tar -cf - --ignore-failed-read /path/to/bad/file's/dir \
> >      /dev/null
> >
> > Possibly -v will also be needed.  In that case, the full scan
>
> should
> > probably write the messages to a file:
> >
> > # tar -cvf - --ignore-failed-read --one-file-system / \
> >      >/dev/null 2>/some/other/volume/tarfiles.txt
> >
> > I don't happen to have any bad blocks to try this on.
> tar seems not to read files that are headed to /dev/null (?)

Lovely.  Well then:

# tar -cvf - --ignore-failed-read --one-file-system / \
      2>/some/other/volume/tarfiles.txt | cat >/dev/null
> trying cat instead. since i am working with hundreds of
> gigabytes... it could take awhile.

Good luck.

[Charles Zeitler]

thanks for the help, Tony.
i finally checked the results of cat'ing to /dev/null,
which found 11 troubled files. having removed them,
am now running smartctl -t on umounted disk.
> # tar -cvf - --ignore-failed-read --one-file-system / \
>      2>/some/other/volume/tarfiles.txt | cat >/dev/null
>
> Good luck.
>
> --

ah... perhaps this will let me check dir files... will try after
the smartctl test.....
> TonyN.:'                       
>      '                              http://www.georgeanelson.com/

charles zeitler

Do What Thou Wilt
        Shall  Be
      The Whole  of
         The Law

-Aleister Crowley

[Charles Zeitler]

On Fri, Oct 16, 2009 at 7:49 PM, charles zeitler  wrote:
> thanks for the help, Tony.
> i finally checked the results of cat'ing to /dev/null,
> which found 11 troubled files. having removed them,
> am now running smartctl -t on umounted disk.
> > # tar -cvf - --ignore-failed-read --one-file-system / \
> >      2>/some/other/volume/tarfiles.txt | cat >/dev/null
> >
> > Good luck.
> >
> > --

second try  of smartctl -t  yielded the same results-
i may have removed    files from 11 bad blocks-
and missed the first.

am now running the tar seq. above (fingers crossed)
> > TonyN.:'                       
> >      '                              http://www.georgeanelson.com/

charles zeitler

[Robert Nichols]

charles zeitler wrote:
> On Fri, Oct 16, 2009 at 7:49 PM, charles zeitler  wrote:
> > thanks for the help, Tony.
> > i finally checked the results of cat'ing to /dev/null,
> > which found 11 troubled files. having removed them,
> > am now running smartctl -t on umounted disk.
> > > # tar -cvf - --ignore-failed-read --one-file-system / \
> > >      2>/some/other/volume/tarfiles.txt | cat >/dev/null
> > >
> > > Good luck.
> > >
> > > --
>
> second try  of smartctl -t  yielded the same results-
> i may have removed    files from 11 bad blocks-
> and missed the first.
>
> am now running the tar seq. above (fingers crossed)

SMART has no idea whether a sector has been allocated to a file or
not.  The test will continue to find the same bad sectors until
writes to those sectors occur, which will allow the drive to remap
them to spare sectors.

[Tony Nelson]

On 09-10-17 01:14:42, charles zeitler wrote:
> On Fri, Oct 16, 2009 at 7:49 PM, charles zeitler 
>
> wrote:
> > thanks for the help, Tony.
> > i finally checked the results of cat'ing to /dev/null,
> > which found 11 troubled files. having removed them,
> > am now running smartctl -t on umounted disk.
> > > # tar -cvf - --ignore-failed-read --one-file-system / \
> > >      2>/some/other/volume/tarfiles.txt | cat >/dev/null
> > >
> > > Good luck.
> > >
> > > --
>
> second try  of smartctl -t  yielded the same results-
> i may have removed    files from 11 bad blocks-
> and missed the first.

Sadly, you have not fixed any of the bad blocks by deleting the files.  
The bad blocks will only be replaced when they are written to.  Now 
that the files are deleted, you will need to write to all the free 
space of the drive to write to the defective blocks.  This should do 
the trick (as root, or from the Rescue CD):

# cp /dev/zero junk ; rm junk
> am now running the tar seq. above (fingers crossed)

I don't expect it to find any defective files, as you already deleted 
them.

[Bruno Wolff III]

On Sat, Oct 17, 2009 at 12:17:36 -0400,
  Tony Nelson  wrote:
> Sadly, you have not fixed any of the bad blocks by deleting the files.  
> The bad blocks will only be replaced when they are written to.  Now 
> that the files are deleted, you will need to write to all the free 
> space of the drive to write to the defective blocks.  This should do 
> the trick (as root, or from the Rescue CD):

He should be able to get the sector numbers by running long self tests.
On the disks I have own, this only gets you one sector per scan rewrite
cycle. Also note that linux tools such as dd write 8 sectors at a time
and if you try to write less than that, a read will be tried first which
will most likely fail because of the bad sector.

[Tony Nelson]

On 09-10-17 14:35:45, Bruno Wolff III wrote:
> On Sat, Oct 17, 2009 at 12:17:36 -0400,
>   Tony Nelson  wrote:
> > Sadly, you have not fixed any of the bad blocks by deleting the
> > files.  The bad blocks will only be replaced when they are written 
> > to.  Now that the files are deleted, you will need to write to all 
> > the free space of the drive to write to the defective blocks.  This 
> > should do the trick (as root, or from the Rescue CD):
>
> He should be able to get the sector numbers by running long self
> tests.  On the disks I have own, this only gets you one sector per 
> scan rewrite cycle.

He gets some of them, then the test aborts.  He needs to fix at least 
those sectors in order to proceed with the test.
> Also note that linux tools such as dd write 8 sectors 
> at a time and if you try to write less than that, a read will be 
> tried first which will most likely fail because of the bad sector.

I don't think he uses dd.  As all the defective files have been 
deleted, the command I gave should write all the bad sectors in about 
the time to do a long test, even bad sectors that haven't been reported 
yet due to the self-test aborting (though I should have said "rm -f").

[Charles Zeitler]

thanks for sticking with me, Tony.
> > On Sat, Oct 17, 2009 at 12:17:36 -0400,
> >   Tony Nelson d
>
> yet due to the self-test aborting (though I should have said "rm -f").

am now running the new, improved
'cp /dev/zero'....

charles zeitler

[Charles Zeitler]

On Sat, Oct 17, 2009 at 9:17 PM, charles zeitler  wrote:
> thanks for sticking with me, Tony.
> > > On Sat, Oct 17, 2009 at 12:17:36 -0400,
> > >   Tony Nelson d
> >
> > yet due to the self-test aborting (though I should have said "rm -f").
>
> am now running the new, improved
> 'cp /dev/zero'....

done. 'current_pending' reduced to 1.
trying a long test...

[Charles Zeitler]

> > am now running the new, improved
> > 'cp /dev/zero'....
>
> done. 'current_pending' reduced to 1.
> trying a long test...

done. still showing the 'read failure', on the same block,
with 90% unchecked...

i'll try to work out something to check bad dir files...

charles zeitler

[Tony Nelson]

On 09-10-18 17:34:11, charles zeitler wrote:
> > > am now running the new, improved
> > > 'cp /dev/zero'....
> >
> > done. 'current_pending' reduced to 1.
> > trying a long test...
>
> done. still showing the 'read failure', on the same block,
> with 90% unchecked...
>
> i'll try to work out something to check bad dir files...

Or maybe back up and restore, possibly with the manufacturer's low-
level format in between?

I wouldn't expect fsck to succeed if there were a bad dir file.

[Charles Zeitler]

On Mon, Oct 19, 2009 at 5:14 PM, Tony Nelson
 wrote:
> On 09-10-18 17:34:11, charles zeitler wrote:
> > > > am now running the new, improved
> > > > 'cp /dev/zero'....
> > >
> > > done. 'current_pending' reduced to 1.
> > > trying a long test...
> >
> > done. still showing the 'read failure', on the same block,
> > with 90% unchecked...
> >
> > i'll try to work out something to check bad dir files...
>
> Or maybe back up and restore, possibly with the manufacturer's low-
> level format in between?
>
> I wouldn't expect fsck to succeed if there were a bad dir file.

u huh. well , apparently, the block that's causing the 'read failure' isn't
being used by a (normal) file... but, i'm planning on an upgrade anyways,
that'll be an opportunity  to clean things up.

thanks for guiding me in the use of smartctl & /dev/zero. stuff i'll
probably need again.
> --
> TonyN.:'                       
>      '                              http://www.georgeanelson.com/

[Tony Nelson]

On 09-10-20 03:03:25, charles zeitler wrote:
> On Mon, Oct 19, 2009 at 5:14 PM, Tony Nelson
>  wrote:

...
> > I wouldn't expect fsck to succeed if there were a bad dir file.
>
> u huh. well , apparently, the block that's causing the 'read failure'
> isn't being used by a (normal) file... but, i'm planning on an 
> upgrade anyways, that'll be an opportunity  to clean things up.
>
> thanks for guiding me in the use of smartctl & /dev/zero. stuff i'll
> probably need again.

You're welcome, and good luck.

[Charles Zeitler]

again, thanks for the replies....
i discovered that smartctl adds the results
of tests to the _top_ of the list, ( duh!) & now
reports a clean sweep...

charles zeitler