[FC3] Oomkiller running amok?

Rhesa Rozendaal perl at rhesa.com
Sun Nov 6 08:11:49 UTC 2005


Hi,

Once in a long while, oomkiller seems to decide my server is out of memory and 
starts killing off processes, until I reboot. This machine runs stable most of 
the time, so it's pretty confusing. I have the same issue occasionally on 
another similar box. RAM consumption never goes over 3GB (out of 4GB). Swap 
usage is virtually zero, and there's plenty of free disk space. Average load 
is around 1.2.

I've been googling quite a bit, and the only relevant looking url was 
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=837184&admit=-682735245+1131261645843+28353475
Unfortunately it was completely useless.

Can anyone give me some hints as to what might be causing this, and what I can 
do about it?

TIA,
Rhesa Rozendaal


Hardware:
Dual Xeon 3.0GHz
4GB RAM
2x 100GB disk in software raid 0

Kernel version:
Linux version 2.6.10-1.760_FC3smp (bhcompile at bugs.build.redhat.com) (gcc 
version 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)) #1 SMP Wed Feb 2 00:29:03 EST 2005

Oomkiller log message example:

Nov  5 18:15:20 web-fed-1 kernel: oom-killer: gfp_mask=0xd0 
 

Nov  5 18:15:20 web-fed-1 kernel: Mem-info: 
 

Nov  5 18:15:20 web-fed-1 kernel: DMA per-cpu: 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 0 hot: low 2, high 6, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 0 cold: low 0, high 2, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 1 hot: low 2, high 6, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 1 cold: low 0, high 2, batch 1 
 
                                                              Nov  5 18:15:20 
web-fed-1 kernel: cpu 2 hot: low 2, high 6, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 2 cold: low 0, high 2, batch 1 
 
                                                              Nov  5 18:15:20 
web-fed-1 kernel: cpu 3 hot: low 2, high 6, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 3 cold: low 0, high 2, batch 1 
 

Nov  5 18:15:20 web-fed-1 kernel: Normal per-cpu: 
 

Nov  5 18:15:20 web-fed-1 kernel: cpu 0 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 0 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 1 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 1 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 2 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 2 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 3 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 3 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: HighMem per-cpu: 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 0 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 0 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 1 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 1 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 2 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 2 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 3 hot: low 32, high 96, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: cpu 3 cold: low 0, high 32, batch 16 
 

Nov  5 18:15:22 web-fed-1 kernel: 
 

Nov  5 18:15:22 web-fed-1 kernel: Free pages:     1130660kB (1126848kB 
HighMem) 

Nov  5 18:15:22 web-fed-1 kernel: Active:439489 inactive:93772 dirty:0 
writeback:0 unstable:0 free:282665 slab:216511 mapped:101178 pagetables:2553 

Nov  5 18:15:22 web-fed-1 kernel: DMA free:68kB min:68kB low:84kB high:100kB 
active:4kB inactive:0kB present:16384kB pages_scanned:2 all_unreclaimable? no 

Nov  5 18:15:22 web-fed-1 kernel: protections[]: 0 0 0 
 

Nov  5 18:15:22 web-fed-1 kernel: Normal free:3744kB min:3756kB low:4692kB 
high:5632kB active:116kB inactive:456kB present:901120kB pages_scanned:910 
all_unreclaimable? yes
Nov  5 18:15:22 web-fed-1 kernel: protections[]: 0 0 0 
 

Nov  5 18:15:22 web-fed-1 kernel: HighMem free:1126848kB min:512kB low:640kB 
high:768kB active:1757836kB inactive:374632kB present:3407872kB 
pages_scanned:0 all_unreclaimable? no
Nov  5 18:15:22 web-fed-1 kernel: protections[]: 0 0 0 
 

Nov  5 18:15:22 web-fed-1 kernel: DMA: 1*4kB 0*8kB 0*16kB 0*32kB 1*64kB 
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 68kB 

Nov  5 18:15:22 web-fed-1 kernel: Normal: 0*4kB 6*8kB 1*16kB 1*32kB 1*64kB 
0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3744kB 

Nov  5 18:15:22 web-fed-1 kernel: HighMem: 1122*4kB 14951*8kB 10664*16kB 
15430*32kB 4673*64kB 299*128kB 4*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 
1126848kB
Nov  5 18:15:22 web-fed-1 kernel: Swap cache: add 57479, delete 57400, find 
30261/33726, race 0+0 

Nov  5 18:15:22 web-fed-1 kernel: Free swap:       8385360kB 
 

Nov  5 18:15:22 web-fed-1 kernel: 1081344 pages of RAM 
 

Nov  5 18:15:22 web-fed-1 kernel: 819184 pages of HIGHMEM 
 

Nov  5 18:15:22 web-fed-1 kernel: 42639 reserved pages 
 

Nov  5 18:15:22 web-fed-1 kernel: 271512 pages shared 
 

Nov  5 18:15:22 web-fed-1 kernel: 79 pages swap cached 
 

Nov  5 18:15:22 web-fed-1 kernel: Out of Memory: Killed process 9456 (httpd). 
 




More information about the users mailing list