Hi all,
I have a strange recurring problem on some of my servers, maybe someone
can help me figure out what might be causing it.
After running a MySQL machine with fairly high load for a while
(~month), RAM usage in top stops making sense. Normally, RES column
accounts for everything currently present in RAM (not swap) and
corresponds pretty well with mem_used-buffers-cache. This is always the
case after a fresh boot and seems to be the case on most other servers.
But that's not the case here:
59098540k used - 1633832k cached - 25824k buffers = 57438884 (54.7G)
supposedly used by all processes
sum(RES column from top) ~= 35127296k (33.5G)
So where's the other 21G??
I'm pretty sure I'm not nitpicking here, this is >30% of total system
RAM unaccounted for. I've tried stopping all non-OS specific processes
(and restarting some services that seemed to eat more RAM that they
should have (irqbalance)) -- and memory is not reclaimed.
Over the few years that I've seen this problem, I've already replaced
all the hardware and upgraded Fedora from 8 to 14 (currently running
2.6.35.14-95.fc14.x86_64) and MySQL and other code without any sign of
improvement.
Interestingly, another machine with same hardware & OS runs MySQL in
slave mode to replicate the DB -- that machine has uptime of 134 days
and does not exhibit the same symptoms. In fact, here is its mem footprint:
63166496k used - 19786032k cached - 2881484k buffers = 40498980k (38.6G)
sum(RES column from top) ~= 45.5G (which makes sense since a few
RAM-hungry processes share memory)
Please help!