filesystem error

[Jeffrey Ross]

I have the following error reported in my logs, I've had it for quite 
some time and through several reboots

EXT4-fs (md2): error count: 1
EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode 1308162
EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
  

md2 is my root partition, I ran "find / -mount -inum 1308162" find 
didn't return any value.

How can I track down this error, identify what file it refers to and 
clear the error out?  if it matters, its an older Fedora system (16 to 
be exact)

Thanks, Jeff

[maderios]

On 04/22/2016 02:39 PM, Jeffrey Ross wrote:
>  I have the following error reported in my logs, I've had it for
> quite
>  some time and through several reboots
>
>  EXT4-fs (md2): error count: 1
>  EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode 1308162
>  EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
>
>
>  md2 is my root partition, I ran "find / -mount -inum 1308162" find
>  didn't return any value.
>
>  How can I track down this error, identify what file it refers to and
>  clear the error out?  if it matters, its an older Fedora system (16 to
>  be exact) 
Hi
'md2' is not a partition name. I don't understand...
Anyway, you can't repair mounted partition.
You must use livecd to repair your / partition with e2fsck
Example if your / partition is /dev/sda3
In a console as root:
e2sfsck -f /dev/sda3

man e2fsck
-f     Force checking even if the file system seems clean
-- 
Maderios

[Samuel Sieb]

On 04/22/2016 07:26 AM, maderios wrote:
>  On 04/22/2016 02:39 PM, Jeffrey Ross wrote:
> > I have the following error reported in my logs, I've had it for quite
> > some time and through several reboots
> >
> > EXT4-fs (md2): error count: 1
> > EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode
> > 1308162
> > EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
> >
> >
> > md2 is my root partition, I ran "find / -mount -inum 1308162" find
> > didn't return any value.
> >
> > How can I track down this error, identify what file it refers to and
> > clear the error out?  if it matters, its an older Fedora system (16 to
> > be exact)
>  Hi
>  'md2' is not a partition name. I don't understand... 
It's a RAID device.

>  Anyway, you can't repair mounted partition.
>  You must use livecd to repair your / partition with e2fsck
>  Example if your / partition is /dev/sda3
>  In a console as root:
>  e2sfsck -f /dev/sda3
>  
That's a good method.  However, you can run safely run e2fsck on a 
mounted filesystem in read-only mode.  This can give some 
false-positives, but it might be able to give you an idea of what's wrong.

e2fsck -fn /dev/sda3

-f to force it, -n to do it-read-only

[Chris Murphy]

On Fri, Apr 22, 2016 at 12:54 PM, Samuel Sieb <samuel(a)sieb.net&gt; wrote:
>  On 04/22/2016 07:26 AM, maderios wrote:
> >
> > On 04/22/2016 02:39 PM, Jeffrey Ross wrote:
> >>
> >> I have the following error reported in my logs, I've had it for quite
> >> some time and through several reboots
> >>
> >> EXT4-fs (md2): error count: 1
> >> EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode
> >> 1308162
> >> EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
> >>
> >>
> >> md2 is my root partition, I ran "find / -mount -inum 1308162" find
> >> didn't return any value.
> >>
> >> How can I track down this error, identify what file it refers to and
> >> clear the error out?  if it matters, its an older Fedora system (16 to
> >> be exact)
> >
> > Hi
> > 'md2' is not a partition name. I don't understand...
>
>
>  It's a RAID device.
>
> > Anyway, you can't repair mounted partition.
> > You must use livecd to repair your / partition with e2fsck
> > Example if your / partition is /dev/sda3
> > In a console as root:
> > e2sfsck -f /dev/sda3
> >
>  That's a good method.  However, you can run safely run e2fsck on a mounted
>  filesystem in read-only mode.  This can give some false-positives, but it
>  might be able to give you an idea of what's wrong.
>
>  e2fsck -fn /dev/sda3
>
>  -f to force it, -n to do it-read-only 

In the OP's example where an md2 device was affected, it needs fsck
run on /dev/md2 (assembled and running but not mounted). Since md2 is
his root fs, that means a.) boot from alternate media or b.) use boot
parameter rd.break=pre-mount and do it at the dracut shell, but this
might require manually running the array, I'm not sure whether it will
be running at this point but it probably won't be mounted (check
first).


-- 
Chris Murphy

[Samuel Sieb]

On 04/22/2016 12:19 PM, Chris Murphy wrote:
>  On Fri, Apr 22, 2016 at 12:54 PM, Samuel Sieb <samuel(a)sieb.net&gt;
> wrote:
> > e2fsck -fn /dev/sda3
> >
> > -f to force it, -n to do it-read-only
>
>  In the OP's example where an md2 device was affected, it needs fsck
>  run on /dev/md2 (assembled and running but not mounted). Since md2 is
>  his root fs, that means a.) boot from alternate media or b.) use boot
>  parameter rd.break=pre-mount and do it at the dracut shell, but this
>  might require manually running the array, I'm not sure whether it will
>  be running at this point but it probably won't be mounted (check
>  first).
>  
Sorry, I was copying the previous poster and forgot the device was 
specified earlier.
The command to run is "e2fsck -fn /dev/md2".  That can be run while it's 
mounted to give an idea of what's wrong with the filesystem.

[Chris Murphy]

On Fri, Apr 22, 2016 at 1:23 PM, Samuel Sieb <samuel(a)sieb.net&gt; wrote:
>  On 04/22/2016 12:19 PM, Chris Murphy wrote:
> >
> > On Fri, Apr 22, 2016 at 12:54 PM, Samuel Sieb <samuel(a)sieb.net&gt; wrote:
> >>
> >> e2fsck -fn /dev/sda3
> >>
> >> -f to force it, -n to do it-read-only
> >
> >
> > In the OP's example where an md2 device was affected, it needs fsck
> > run on /dev/md2 (assembled and running but not mounted). Since md2 is
> > his root fs, that means a.) boot from alternate media or b.) use boot
> > parameter rd.break=pre-mount and do it at the dracut shell, but this
> > might require manually running the array, I'm not sure whether it will
> > be running at this point but it probably won't be mounted (check
> > first).
> >
>  Sorry, I was copying the previous poster and forgot the device was specified
>  earlier.
>  The command to run is "e2fsck -fn /dev/md2".  That can be run while it's
>  mounted to give an idea of what's wrong with the filesystem. 

I would not exclusively trust user space messages. Check dmesg before
and after for the appearance of new messages that may not even be
directly ext4 related. This could be a case of a bad sector
instigating the problem.


-- 
Chris Murphy

[Jon LaBadie]

On Fri, Apr 22, 2016 at 08:39:05AM -0400, Jeffrey Ross wrote:
>  I have the following error reported in my logs, I've had it for
> quite some
>  time and through several reboots
>  
>  EXT4-fs (md2): error count: 1
>  EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode 1308162
>  EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
>  
>  md2 is my root partition, I ran "find / -mount -inum 1308162" find didn't
>  return any value.
>  
>  How can I track down this error, identify what file it refers to and clear
>  the error out?  if it matters, its an older Fedora system (16 to be exact) 
The inode may not be allocated to a file and thus available for allocation.

UNIX systems had a program fsdb (file system debugger) that let you
view and edit components of a file system.  Is there a similar program
for ext-X file systems?

Jon
-- 
Jon H. LaBadie                  jonfu(a)jgcomp.com

[Jeffrey Ross]

>  I have the following error reported in my logs, I've had it for
> quite
>  some time and through several reboots
>
>  EXT4-fs (md2): error count: 1
>  EXT4-fs (md2): initial error at 1452554514: ext4_lookup:1343: inode
>  1308162
>  EXT4-fs (md2): last error at 1452554514: ext4_lookup:1343: inode 1308162
>
>
>  md2 is my root partition, I ran "find / -mount -inum 1308162" find
>  didn't return any value.
>
>  How can I track down this error, identify what file it refers to and
>  clear the error out?  if it matters, its an older Fedora system (16 to
>  be exact)
>
>  Thanks, Jeff 

Thanks everybody for the suggestions, I figured running fsck would fix it,
I should have mentioned that in the past I've attempted to run fsck but it
hadn't cleared, probably how I ran it.

running fsck -fn /dev/md2 does in fact produce a handful of errors "inode
xxx ref count is 2 should be 1" plus a few other errors.



Since the machine is remote from me I'd like to issue a reboot and force
the disk to clean (I do have access to the system, just need to make a few
calls).

So my question is, after doing a little bit of searching can somebody
confirm this approach?

I thought (possibly incorrectly) I could force a fsck on reboot by
touching the file "/forcefsck"  but searching further I think I will also
need to create the file "/etc/sysconfig/autofsck" and put these two lines
in:
AUTOFSCK_DEF_CHECK=yes
AUTOFSCK_OPT="-y"

hopefully this would clear the filesystem on a reload.

The other comment I saw would be to put a "-y" in the /forcefsck file

Am I on the right track? I'd prefer to avoid a drive over to the
datacenter and arranging access even though its only 15 minutes from home.

Booting the system on recovery media although not impossible will be
difficult.

Thanks, Jeff

[Samuel Sieb]

On 04/22/2016 01:04 PM, Jeffrey Ross wrote:
>  Thanks everybody for the suggestions, I figured running fsck would
> fix it,
>  I should have mentioned that in the past I've attempted to run fsck but it
>  hadn't cleared, probably how I ran it.
>
>  running fsck -fn /dev/md2 does in fact produce a handful of errors "inode
>  xxx ref count is 2 should be 1" plus a few other errors.
>  
Those are likely harmless, especially since you are running it on a 
mounted fs.  What are the other errors?

>  I thought (possibly incorrectly) I could force a fsck on reboot by
>  touching the file "/forcefsck"  but searching further I think I will also
>  need to create the file "/etc/sysconfig/autofsck" and put these two lines
>  in:
>  AUTOFSCK_DEF_CHECK=yes
>  AUTOFSCK_OPT="-y"
>  
Just touch the file /forcefsck and it will force an fsck on the next reboot.

>  The other comment I saw would be to put a "-y" in the
> /forcefsck file
>  
You can put that option in /fsckoptions, but I'm not sure what 
difference it would make in this case.

[Gordon Messmer]

On 04/22/2016 05:39 AM, Jeffrey Ross wrote:
>  md2 is my root partition, I ran "find / -mount -inum
> 1308162" find 
>  didn't return any value.  
Using debugfs is faster:
# debugfs /dev/md2
debugfs: ncheck 1308162

However, as I understand it, the message you recorded is "strictly for 
information"
https://lkml.org/lkml/2010/9/26/105