A test malloc program makes 64-Bit FC-4 dying

[Jonathan Berry]

On 11/3/05, Reuben D. Budiardja <techlist at pathfinder.phys.utk.edu> wrote:
>
> Hello,
> I have a small program that I use to check malloc. The program supposed to
> exhaust memory (heap) from malloc, then quit. Running this program on a
> 32-Bit RHEL 4, RHEL 3, FC-4, I got the program to terminated pretty quicky
> after malloc cannot allocated more memory.
> However, when I run this program on my desktop: AMD64 FC 4 64-bit, the program
> does not terminate after a while. But it grinds this machine to almost to a
> halt, and I got page-swapping, etc. I finally managed to kill the program.
>
> My question is, why does this happen in FC-4 64-bit ? what is the difference
> on this machine and other machine ? This supposedly should not happend since
> I run this program as regular user, and it seems "dangerous" that a regular
> user can almost bring this machine down by running this.
>
> I checked 'ulimit' on my shell and it's unlimited. But even when I set 'ulimit
> -m 12000', the same thing still happened. On other machines, i didn't have to
> do anything. I've checked also /etc/security/limits.conf and they are all
> default (ie. nothing in it). Below is the program. Any help or info about
> this is appreciated.
>
> Thanks a lot.
>
> ------ kgobble.c----
> #include <stdio.h>
> #include <string.h>
>
> main()
> {
>   int *i;
>   int j=1;
>   char buf[15];
>
>   printf("Starting kgobble\n");
>
>   while(1)
>   {
>     i=(int *)malloc(8176 * j);
>     if (i==0)
>     {
>       write(1,"Memory exhaustion complete\n",27);
>       exit(0);
>     }
>     sprintf(buf,"%x\n",i);
>     write(1,buf,strlen(buf));
>     bzero(buf,15);
>     j++;
>   }
> }
>
>
> RDB

Hi Reuben,

Question: At what value of i does the above program die when on a
32-bit machine?  Is it the same, or does it vary between machines
(different hardware)?  One difference to note between 32- and 64-bit
Linux is that 64-bit has a *much* (this is an understatement) larger
virtual memory space.  Sometimes Linux and malloc can do some
interesting things with allocating memory, especially when you don't
actually use the memory you are allocating.  I haven't looked into
this deeply, so I don't know the details.  I think a term looking into
is memory overcommit.  What may be happening is that the kernel is
allowing you to allocate memory as long as it fits in your virtual
memory, even though you don't have the physical memory available
(whether RAM or swap).
With 32-bit, you only (heh, only, how things have changed...) have 4
GB of virtual memory space.  With 64-bit, I think you get something on
the order of 16 TB of virtual memory space (I know it doesn't use the
whole 64-bits worth of address, which would be 16 exabytes (giga,
tera, peta, exa)).  Or maybe it is you can have 16 TB of physical RAM
and much more virtual memory space (a 48-bit address seems to stick in
my mind...).  Anyway, if your program is only stopping because it is
filling up it's virtual memory, then, well, you are going to wait a
long time for your program to stop in 64-bit, to say the least.
My initial response to your email was that you are seeing a virtual
memory difference here, whether or not my speculation is completely
accurate, I think this at least has something to do with it.  What
exactly are you trying to accomplish with this program?  I guess you
are trying to make sure that someone cannot pull a "fork bomb" like
resource DoS attack?

Jonathan