f16 :: opteron 6140 :: 3.2.3-2.fc16.x86_64 :: kernel BUG: soft lockup - CPU#2 stuck for 22s! [khugepaged:82]

[Rick Stevens]

On 02/09/2012 01:51 AM, Adrian Sevcenco wrote:
> Hi guys! I keep having some stops and blockages in the context of $subj..
> Any idea about the problem?

It is not a software problem--it's a CPU or motherboard (mobo)
problem.  We've had those on several of our servers (all of them
SuperMicro servers such as yours as a matter of fact).  I believe one
was an actual bad CPU, the others were failures in the CPU support
logic on the mobo (something in the interrupt routing system IIRC).

I'd contact SuperMicro if you're still under warranty.  It's a hardware
fault and you'll have to sort out how you're going to approach it.

> [91464.173992] BUG: soft lockup - CPU#2 stuck for 22s! [khugepaged:82]
> [91464.173998] Modules linked in: binfmt_misc ppdev parport_pc lp
> parport nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables
> nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack joydev nvidia(P)
> snd_hda_codec_hdmi microcode sp5100_tco i2c_piix4 serio_raw k10temp
> uinput edac_core edac_mce_amd snd_hda_intel snd_hda_codec snd_hwdep
> snd_seq snd_seq_device snd_pcm snd_timer snd i2c_core soundcore
> snd_page_alloc e1000e [last unloaded: scsi_wait_scan]
> [91464.174022] CPU 2
> [91464.174022] Modules linked in: binfmt_misc ppdev parport_pc lp
> parport nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables
> nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack joydev nvidia(P)
> snd_hda_codec_hdmi microcode sp5100_tco i2c_piix4 serio_raw k10temp
> uinput edac_core edac_mce_amd snd_hda_intel snd_hda_codec snd_hwdep
> snd_seq snd_seq_device snd_pcm snd_timer snd i2c_core soundcore
> snd_page_alloc e1000e [last unloaded: scsi_wait_scan]
> [91464.174022]
> [91464.174022] Pid: 82, comm: khugepaged Tainted: P           O
> 3.2.3-2.fc16.x86_64 #1 Supermicro H8SGL/H8SGL
> [91464.174022] RIP: 0010:[<ffffffff812c5126>]  [<ffffffff812c5126>]
> __bitmap_empty+0x6/0x80
> [91464.174022] RSP: 0018:ffff880410e01d10  EFLAGS: 00000206
> [91464.174022] RAX: 0000000000000000 RBX: ffffc90001812048 RCX:
> 000000000000013f
> [91464.174022] RDX: 0000000000000100 RSI: 0000000000000100 RDI:
> ffffffff81cb1318
> [91464.174022] RBP: ffff880410e01d50 R08: ffffffff81cb1338 R09:
> 0000000000000100
> [91464.174022] R10: 0000000000000201 R11: 0000000000000000 R12:
> ffff880410e01ce0
> [91464.174022] R13: 0000000000000286 R14: ffffc90001812000 R15:
> ffffea0007408000
> [91464.174022] FS:  00007f15554929c0(0000) GS:ffff88042fc40000(0000)
> knlGS:0000000000000000
> [91464.174022] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [91464.174022] CR2: 00007f1553036000 CR3: 0000000001a05000 CR4:
> 00000000000006e0
> [91464.174022] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [91464.174022] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [91464.174022] Process khugepaged (pid: 82, threadinfo ffff880410e00000,
> task ffff880410daae40)
> [91464.174022] Stack:
> [91464.174022]  ffff880410e01d50 ffffffff8104476f ffff880410e01d80
> ffffffff81172e0e
> [91464.174022]  ffff88040fd95b00 ffff88040fd95dc8 0000000000000201
> ffff88040fd95b00
> [91464.174022]  ffff880410e01d70 ffffffff8104484b ffff88020e330b00
> 0000000383bb2067
> [91464.174022] Call Trace:
> [91464.174022]  [<ffffffff8104476f>] ? native_flush_tlb_others+0x11f/0x130
> [91464.174022]  [<ffffffff81172e0e>] ? mem_cgroup_charge_common+0xae/0xe0
> [91464.174022]  [<ffffffff8104484b>] flush_tlb_mm+0x5b/0xa0
> [91464.174022]  [<ffffffff8114eaa0>] pmdp_clear_flush+0x30/0x40
> [91464.174022]  [<ffffffff8116aad3>] khugepaged+0xd33/0x1380
> [91464.174022]  [<ffffffff81090440>] ? remove_wait_queue+0x50/0x50
> [91464.174022]  [<ffffffff81169da0>] ? collect_mm_slot+0xa0/0xa0
> [91464.174022]  [<ffffffff8108fb9c>] kthread+0x8c/0xa0
> [91464.174022]  [<ffffffff815eb8f4>] kernel_thread_helper+0x4/0x10
> [91464.174022]  [<ffffffff8108fb10>] ? kthread_worker_fn+0x190/0x190
> [91464.174022]  [<ffffffff815eb8f0>] ? gs_change+0x13/0x13
> [91464.174022] Code: f0 48 89 45 c0 48 8d 45 d0 4c 89 4d f8 c7 45 b8 10
> 00 00 00 48 89 45 c8 e8 38 ff ff ff c9 c3 90 90 90 90 90 90 8d 4e 3f 85
> f6 55<0f>  49 ce 48 89 e5 c1 f9 06 85 c9 7e 61 31 c0 48 83 3f 00 75 52
> [91464.174022] Call Trace:
> [91464.174022]  [<ffffffff8104476f>] ? native_flush_tlb_others+0x11f/0x130
> [91464.174022]  [<ffffffff81172e0e>] ? mem_cgroup_charge_common+0xae/0xe0
> [91464.174022]  [<ffffffff8104484b>] flush_tlb_mm+0x5b/0xa0
> [91464.174022]  [<ffffffff8114eaa0>] pmdp_clear_flush+0x30/0x40
> [91464.174022]  [<ffffffff8116aad3>] khugepaged+0xd33/0x1380
> [91464.174022]  [<ffffffff81090440>] ? remove_wait_queue+0x50/0x50
> [91464.174022]  [<ffffffff81169da0>] ? collect_mm_slot+0xa0/0xa0
> [91464.174022]  [<ffffffff8108fb9c>] kthread+0x8c/0xa0
> [91464.174022]  [<ffffffff815eb8f4>] kernel_thread_helper+0x4/0x10
> [91464.174022]  [<ffffffff8108fb10>] ? kthread_worker_fn+0x190/0x190
> [91464.174022]  [<ffffffff815eb8f0>] ? gs_change+0x13/0x13

----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital    ricks at alldigital.com -
- AIM/Skype: therps2        ICQ: 22643734            Yahoo: origrps2 -
-                                                                    -
-      Batteries not included.  Offer not valid in some states.      -
-           Your mileage may vary.  Void where prohibited.           -
----------------------------------------------------------------------