no fsck for unmounted filesystems

[Reindl Harald]

/dev/sda1 is a ordinary partition containing only /boot
why does the kernel report it is in use?
if that would be true it could not be unmounted
___________________________________________

[root@asterisk:~]$ /usr/sbin/fsck.ext4 -f /dev/sda1
e2fsck 1.42.12 (29-Aug-2014)
/dev/sda1 is in use.
e2fsck: Cannot continue, aborting.
___________________________________________

[root@asterisk:~]$ lsof | grep /dev/sda1
[root@asterisk:~]$
___________________________________________

[root@asterisk:~]$ /usr/sbin/fsck.ext4 -n /dev/sda1
e2fsck 1.42.12 (29-Aug-2014)
Warning!  /dev/sda1 is in use.
boot was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (454751, counted=469570).
Fix? no

Free inodes count wrong (130215, counted=130220).
Fix? no

boot: 345/130560 files (22.6% non-contiguous), 66464/521215 blocks

[Eric Sandeen]

On 2/16/15 4:30 AM, Reindl Harald wrote:
>  /dev/sda1 is a ordinary partition containing only /boot
>  why does the kernel report it is in use?
>  if that would be true it could not be unmounted
>  ___________________________________________
>  
>  [root@asterisk:~]$ /usr/sbin/fsck.ext4 -f /dev/sda1
>  e2fsck 1.42.12 (29-Aug-2014)
>  /dev/sda1 is in use.
>  e2fsck: Cannot continue, aborting. 
I just saw a similar report on #ext4; do you still have a jbd
thread running for this filesystem?

Also, in the spirit of useful bug reporting, what kernel
are you running?

Can you run strace of e2fsck, and see if an open(O_EXCL) fails,
or find some other hint about how e2fsck decided it was in
use?  Possibly paste the last couple hundred lines of strace
somewhere.

This may be better suited for a bug report, or a thread on
linux-ext4, than the fedora-kernel list.

FWIW, the errors below do look indicative of a filesystem
that has never had the superblock written out during an
actual unmount...

Thanks,
-Eric

________________________________________
>  
>  [root@asterisk:~]$ lsof | grep /dev/sda1
>  [root@asterisk:~]$
>  ___________________________________________
>  
>  [root@asterisk:~]$ /usr/sbin/fsck.ext4 -n /dev/sda1
>  e2fsck 1.42.12 (29-Aug-2014)
>  Warning!  /dev/sda1 is in use.
>  boot was not cleanly unmounted, check forced.
>  Pass 1: Checking inodes, blocks, and sizes
>  Pass 2: Checking directory structure
>  Pass 3: Checking directory connectivity
>  Pass 4: Checking reference counts
>  Pass 5: Checking group summary information
>  Free blocks count wrong (454751, counted=469570).
>  Fix? no
>  
>  Free inodes count wrong (130215, counted=130220).
>  Fix? no
>  
>  boot: 345/130560 files (22.6% non-contiguous), 66464/521215 blocks
>  
>  
>  
>  _______________________________________________
>  kernel mailing list
>  kernel(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/kernel
>

[Reindl Harald]

Am 16.02.2015 um 16:50 schrieb Eric Sandeen:
>  On 2/16/15 4:30 AM, Reindl Harald wrote:
> > /dev/sda1 is a ordinary partition containing only /boot
> > why does the kernel report it is in use?
> > if that would be true it could not be unmounted
> > ___________________________________________
> >
> > [root@asterisk:~]$ /usr/sbin/fsck.ext4 -f /dev/sda1
> > e2fsck 1.42.12 (29-Aug-2014)
> > /dev/sda1 is in use.
> > e2fsck: Cannot continue, aborting.
>
>  I just saw a similar report on #ext4; do you still have a jbd
>  thread running for this filesystem? 
this is a ext2 filesystem

[root@mail-gw:~]$ ps aux | grep jbd
root       203  0.0  0.0      0     0 ?        S    Feb15   0:00 
[jbd2/sdb1-8]
root       343  0.0  0.0      0     0 ?        S    Feb15   0:00 
[jbd2/sdc1-8]
root       347  0.0  0.0      0     0 ?        S    Feb15   0:00 
[jbd2/sdd1-8]
root     20398  0.0  0.0 112688  2228 pts/2    R<+  16:56   0:00 
/usr/bin/grep --color jbd

[root@mail-gw:~]$ df
Dateisystem    Typ  Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sdb1      ext4  5,8G    1,6G  4,2G   28% /
/dev/sdc1      ext4  4,0G    493M  3,5G   13% /var/log
/dev/sdd1      ext4  2,0G    4,4M  2,0G    1% /var/spool
/dev/sda1      ext4  493M     36M  458M    8% /boot


>  Also, in the spirit of useful bug reporting, what kernel
>  are you running? 
always the lastest fedora kernel
3.18.7-100.fc20.x86_64

>  Can you run strace of e2fsck, and see if an open(O_EXCL) fails,
>  or find some other hint about how e2fsck decided it was in
>  use?  Possibly paste the last couple hundred lines of strace
>  somewhere. 
i will try that on a non-production machine

>  This may be better suited for a bug report, or a thread on
>  linux-ext4, than the fedora-kernel list. 
yes, my intention was to first ask if the problem is known and maybe 
even a solution inthe pipeline and if not finally catch the needed infos 
like the strace in the initial bugreport

after it's created i will respond to this post with the link

>  FWIW, the errors below do look indicative of a filesystem
>  that has never had the superblock written out during an
>  actual unmount... 
likely because the -n param

the corruption itself may come from 
https://bugzilla.redhat.com/show_bug.cgi?id=1192834 which is also really 
strange

>  ________________________________________
> >
> > [root@asterisk:~]$ lsof | grep /dev/sda1
> > [root@asterisk:~]$
> > ___________________________________________
> >
> > [root@asterisk:~]$ /usr/sbin/fsck.ext4 -n /dev/sda1
> > e2fsck 1.42.12 (29-Aug-2014)
> > Warning!  /dev/sda1 is in use.
> > boot was not cleanly unmounted, check forced.
> > Pass 1: Checking inodes, blocks, and sizes
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > Pass 5: Checking group summary information
> > Free blocks count wrong (454751, counted=469570).
> > Fix? no
> >
> > Free inodes count wrong (130215, counted=130220).
> > Fix? no
> >
> > boot: 345/130560 files (22.6% non-contiguous), 66464/521215 blocks

[Eric Sandeen]

On 2/16/15 10:00 AM, Reindl Harald wrote:
>  
>  
>  Am 16.02.2015 um 16:50 schrieb Eric Sandeen:
> > On 2/16/15 4:30 AM, Reindl Harald wrote:
> >> /dev/sda1 is a ordinary partition containing only /boot
> >> why does the kernel report it is in use?
> >> if that would be true it could not be unmounted
> >> ___________________________________________
> >>
> >> [root@asterisk:~]$ /usr/sbin/fsck.ext4 -f /dev/sda1
> >> e2fsck 1.42.12 (29-Aug-2014)
> >> /dev/sda1 is in use.
> >> e2fsck: Cannot continue, aborting.
> >
> > I just saw a similar report on #ext4; do you still have a jbd
> > thread running for this filesystem?
>  
>  this is a ext2 filesystem
>  
>  [root@mail-gw:~]$ ps aux | grep jbd
>  root       203  0.0  0.0      0     0 ?        S    Feb15   0:00 [jbd2/sdb1-8]
>  root       343  0.0  0.0      0     0 ?        S    Feb15   0:00 [jbd2/sdc1-8]
>  root       347  0.0  0.0      0     0 ?        S    Feb15   0:00 [jbd2/sdd1-8]
>  root     20398  0.0  0.0 112688  2228 pts/2    R<+  16:56   0:00 /usr/bin/grep --color
> jbd
>  
>  [root@mail-gw:~]$ df
>  Dateisystem    Typ  Größe Benutzt Verf. Verw% Eingehängt auf
>  /dev/sdb1      ext4  5,8G    1,6G  4,2G   28% /
>  /dev/sdc1      ext4  4,0G    493M  3,5G   13% /var/log
>  /dev/sdd1      ext4  2,0G    4,4M  2,0G    1% /var/spool
>  /dev/sda1      ext4  493M     36M  458M    8% /boot
>  
>  
> > Also, in the spirit of useful bug reporting, what kernel
> > are you running?
>  
>  always the lastest fedora kernel
>  3.18.7-100.fc20.x86_64
>  
> > Can you run strace of e2fsck, and see if an open(O_EXCL) fails,
> > or find some other hint about how e2fsck decided it was in
> > use?  Possibly paste the last couple hundred lines of strace
> > somewhere.
>  
>  i will try that on a non-production machine
>  
> > This may be better suited for a bug report, or a thread on
> > linux-ext4, than the fedora-kernel list.
>  
>  yes, my intention was to first ask if the problem is known and maybe even a solution
> inthe pipeline and if not finally catch the needed infos like the strace in the initial
> bugreport
>  
>  after it's created i will respond to this post with the link
>  
> > FWIW, the errors below do look indicative of a filesystem
> > that has never had the superblock written out during an
> > actual unmount...
>  
>  likely because the -n param
>  
>  the corruption itself may come from https://bugzilla.redhat.com/show_bug.cgi?id=1192834
> which is also really strange 
Let's continue the discussion in that bug, if this behavior is a result of the
testcase in that bug.

-Eric

[Reindl Harald]

Am 16.02.2015 um 17:03 schrieb Eric Sandeen:
>  On 2/16/15 10:00 AM, Reindl Harald wrote:
> > the corruption itself may come from
> https://bugzilla.redhat.com/show_bug.cgi?id=1192834 which is also really strange
>
>  Let's continue the discussion in that bug, if this behavior is a result of the
> testcase in that bug 
these are two different things, even without any corruption the result 
is the same - i have here 30 clones using the same /boot and even the 
ones with a clean FS don't allow unmount and fsck

the fact that you mostly can't unmount and/or fsck annoys me over years 
and on raid-devices it's neraly 100% so but that it also affects a ext2 
/boot partition seems to be recent "improvement" of that behavior

if you comment out the /boot entry in /etc/fstab fsck you can call fsck 
without any issue and i would strongly expect the same behavior after 
unmount since if the device really is in use unmount would fail

[Reindl Harald]

see https://bugzilla.redhat.com/show_bug.cgi?id=1193203

most strange thing is that even mount the partition again fails because 
the OS pretends it's already mountd or in use, the issue trying to mount 
again don't happen predictable but i need to reboot that amchine now or 
the next kernel update will end in a tragedy

Am 16.02.2015 um 17:00 schrieb Reindl Harald:
>  Am 16.02.2015 um 16:50 schrieb Eric Sandeen:
> > On 2/16/15 4:30 AM, Reindl Harald wrote:
> >> /dev/sda1 is a ordinary partition containing only /boot
> >> why does the kernel report it is in use?
> >> if that would be true it could not be unmounted
> >> ___________________________________________
> >>
> >> [root@asterisk:~]$ /usr/sbin/fsck.ext4 -f /dev/sda1
> >> e2fsck 1.42.12 (29-Aug-2014)
> >> /dev/sda1 is in use.
> >> e2fsck: Cannot continue, aborting.
> >
> > I just saw a similar report on #ext4; do you still have a jbd
> > thread running for this filesystem?
>
>  this is a ext2 filesystem
>
>  [root@mail-gw:~]$ ps aux | grep jbd
>  root       203  0.0  0.0      0     0 ?        S    Feb15   0:00
>  [jbd2/sdb1-8]
>  root       343  0.0  0.0      0     0 ?        S    Feb15   0:00
>  [jbd2/sdc1-8]
>  root       347  0.0  0.0      0     0 ?        S    Feb15   0:00
>  [jbd2/sdd1-8]
>  root     20398  0.0  0.0 112688  2228 pts/2    R<+  16:56   0:00
>  /usr/bin/grep --color jbd
>
>  [root@mail-gw:~]$ df
>  Dateisystem    Typ  Größe Benutzt Verf. Verw% Eingehängt auf
>  /dev/sdb1      ext4  5,8G    1,6G  4,2G   28% /
>  /dev/sdc1      ext4  4,0G    493M  3,5G   13% /var/log
>  /dev/sdd1      ext4  2,0G    4,4M  2,0G    1% /var/spool
>  /dev/sda1      ext4  493M     36M  458M    8% /boot
>
>
> > Also, in the spirit of useful bug reporting, what kernel
> > are you running?
>
>  always the lastest fedora kernel
>  3.18.7-100.fc20.x86_64
>
> > Can you run strace of e2fsck, and see if an open(O_EXCL) fails,
> > or find some other hint about how e2fsck decided it was in
> > use?  Possibly paste the last couple hundred lines of strace
> > somewhere.
>
>  i will try that on a non-production machine
>
> > This may be better suited for a bug report, or a thread on
> > linux-ext4, than the fedora-kernel list.
>
>  yes, my intention was to first ask if the problem is known and maybe
>  even a solution inthe pipeline and if not finally catch the needed infos
>  like the strace in the initial bugreport
>
>  after it's created i will respond to this post with the link
>
> > FWIW, the errors below do look indicative of a filesystem
> > that has never had the superblock written out during an
> > actual unmount...
>
>  likely because the -n param
>
>  the corruption itself may come from
>  https://bugzilla.redhat.com/show_bug.cgi?id=1192834 which is also really
>  strange
>
> > ________________________________________
> >>
> >> [root@asterisk:~]$ lsof | grep /dev/sda1
> >> [root@asterisk:~]$
> >> ___________________________________________
> >>
> >> [root@asterisk:~]$ /usr/sbin/fsck.ext4 -n /dev/sda1
> >> e2fsck 1.42.12 (29-Aug-2014)
> >> Warning!  /dev/sda1 is in use.
> >> boot was not cleanly unmounted, check forced.
> >> Pass 1: Checking inodes, blocks, and sizes
> >> Pass 2: Checking directory structure
> >> Pass 3: Checking directory connectivity
> >> Pass 4: Checking reference counts
> >> Pass 5: Checking group summary information
> >> Free blocks count wrong (454751, counted=469570).
> >> Fix? no
> >>
> >> Free inodes count wrong (130215, counted=130220).
> >> Fix? no
> >>
> >> boot: 345/130560 files (22.6% non-contiguous), 66464/521215 blocks