F20 Kernel 3.14.x / raid-check and freezes

[Reindl Harald]

Am 01.05.2014 14:12, schrieb Bruno Wolff III:
> On Thu, May 01, 2014 at 11:18:25 +0200,
> Reindl Harald <h.reindl at thelounge.net> wrote:
>> has anybody else faced complete freezes of a machine
>> due raid-check? today the same as last thursday while
>> the freeze seems not to happen by the raid-check itself
>> according to running smokepings from outside which are
>> green until i try to use the machine
> 
> I had something similar happen in pre-release kernels a few versions ago, but it got fixed and I am not seeing it now

interesting - thanks for feedback

i have faced this only on F20 with the recent 3.14 kernel which
is except the weekly raid-check hyperstable under any heavy
networking and/or IO load, what makes it hard to debug is that
it's not triggerable by starting raid-check manually

the only information i sadly can provide is partitioning / raid setup


[root at srv-rhsoft:~]$ df
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/md1       ext4   29G  6.4G   23G  23% /
/dev/md0       ext4  485M   37M  444M   8% /boot
/dev/md2       ext4  3.6T  2.0T  1.6T  57% /mnt/data


[root at srv-rhsoft:~]$ cat /usr/local/bin/raid_status
#!/usr/bin/bash
echo "---------------------------------------------------------------------------------"
/sbin/mdadm --detail /dev/md0
echo "---------------------------------------------------------------------------------"
/sbin/mdadm --detail /dev/md1
echo "---------------------------------------------------------------------------------"
/sbin/mdadm --detail /dev/md2
echo "---------------------------------------------------------------------------------"
cat /proc/mdstat
echo "---------------------------------------------------------------------------------"


[root at srv-rhsoft:~]$ raid_status
---------------------------------------------------------------------------------
/dev/md0:
        Version : 1.0
  Creation Time : Wed Jun  8 13:10:48 2011
     Raid Level : raid1
     Array Size : 511988 (500.07 MiB 524.28 MB)
  Used Dev Size : 511988 (500.07 MiB 524.28 MB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Thu May  1 09:42:22 2014
          State : clean, resyncing (DELAYED)
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

           Name : localhost.localdomain:0
           UUID : 1d691642:baed26df:1d197496:4fb00ff8
         Events : 651

    Number   Major   Minor   RaidDevice State
       0       8       49        0      active sync   /dev/sdd1
       5       8       33        1      active sync   /dev/sdc1
       4       8        1        2      active sync   /dev/sda1
       3       8       17        3      active sync   /dev/sdb1
---------------------------------------------------------------------------------
/dev/md1:
        Version : 1.1
  Creation Time : Wed Jun  8 13:10:52 2011
     Raid Level : raid10
     Array Size : 30716928 (29.29 GiB 31.45 GB)
  Used Dev Size : 15358464 (14.65 GiB 15.73 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Thu May  1 14:43:16 2014
          State : active, resyncing (DELAYED)
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

           Name : localhost.localdomain:1
           UUID : b7475879:c95d9a47:c5043c02:0c5ae720
         Events : 12359

    Number   Major   Minor   RaidDevice State
       0       8       50        0      active sync set-A   /dev/sdd2
       5       8       34        1      active sync set-B   /dev/sdc2
       4       8        2        2      active sync set-A   /dev/sda2
       3       8       18        3      active sync set-B   /dev/sdb2
---------------------------------------------------------------------------------
/dev/md2:
        Version : 1.1
  Creation Time : Wed Jun  8 13:10:56 2011
     Raid Level : raid10
     Array Size : 3875222528 (3695.70 GiB 3968.23 GB)
  Used Dev Size : 1937611264 (1847.85 GiB 1984.11 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Thu May  1 14:43:23 2014
          State : active, checking
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

   Check Status : 42% complete

           Name : localhost.localdomain:2
           UUID : ea253255:cb915401:f32794ad:ce0fe396
         Events : 496336

    Number   Major   Minor   RaidDevice State
       0       8       51        0      active sync set-A   /dev/sdd3
       5       8       35        1      active sync set-B   /dev/sdc3
       4       8        3        2      active sync set-A   /dev/sda3
       3       8       19        3      active sync set-B   /dev/sdb3
---------------------------------------------------------------------------------
Personalities : [raid1] [raid10]
md2 : active raid10 sdd3[0] sda3[4] sdb3[3] sdc3[5]
      3875222528 blocks super 1.1 512K chunks 2 near-copies [4/4] [UUUU]
      [========>............]  check = 42.9% (1665190848/3875222528) finish=278.2min speed=132374K/sec
      bitmap: 3/29 pages [12KB], 65536KB chunk

md1 : active raid10 sda2[4] sdb2[3] sdd2[0] sdc2[5]
      30716928 blocks super 1.1 512K chunks 2 near-copies [4/4] [UUUU]
        resync=DELAYED
      bitmap: 1/1 pages [4KB], 65536KB chunk

md0 : active raid1 sdb1[3] sda1[4] sdd1[0] sdc1[5]
      511988 blocks super 1.0 [4/4] [UUUU]
        resync=DELAYED

unused devices: <none>
---------------------------------------------------------------------------------