Memory issue

[Mike McGrath]

On Sun, 22 Mar 2009, Matt Domsch wrote:

> On Sat, Mar 21, 2009 at 10:26:21PM -0500, Mike McGrath wrote:
> > What am I missing here?  This happened on db3 again -
> >
> >              total       used       free     shared    buffers     cached
> > Mem:      18480632   18383512      97120          0      25968   17305332
> > -/+ buffers/cache:    1052212   17428420
> > Swap:      3145720    3145720          0
> >
> >
> > 1) This box was swapping out like crazy.
> >
> > 2) All swap was full
> >
> > 3) oomkiller didn't start
> >
> > 4) Load went up a little bit, topped out at 3
> >
> > 5) no issues with the box at all, has postgres on it, kept running, wasn't
> > slow, box was completely responsive.
> >
> > 6) It's done this before, and aside from nagios alerting us the swap was
> > full, not had any issues with the box.
>
> There are ~30 postgres processes running, many with RSS > 1.6GB, and
> nearly all marked idle.  If these processes aren't actively running,
> they each can get swapped out to free memory for use by the page
> cache. The page cache is consuming 17GB, which it's free to do if no
> applications are actually _running_ that need more memory.  The
> processes that are running and touching the database will cause those
> disk accesses to get cached in the page cache.
>

Very interesting, I didn't realize that RSS could be swapped out for page
cache on apps that weren't in the run state.

	-Mike