PAE kernel and 4GB of memory

[Howard Wilkinson]

I am looking for a definitive answer to the question of where the PAE 
kernels become useful. I have seen various articles that mention needing 
PAE kernels if you have more then 4GB of physical memory in a 32-bit 
processor environment. I have also seen statements that say you need 
them if you have 4GB or more of memory. Now which is right? Also, even 
if you need a PAE kernel because the last few bytes are not addressable 
when you have exactly 4GB is this useful or is the trade off of larger 
page tables and pages going to eat any benefit of being able to address 
these few bytes and if so when does the PAE kernel become useful?

Howard.

[Bill Davidsen]

Howard Wilkinson wrote:
>  I am looking for a definitive answer to the question of where the PAE
>
>  kernels become useful. I have seen various articles that mention needing 
>  PAE kernels if you have more then 4GB of physical memory in a 32-bit 
>  processor environment. I have also seen statements that say you need 
>  them if you have 4GB or more of memory. Now which is right? Also, even 
>  if you need a PAE kernel because the last few bytes are not addressable 
>  when you have exactly 4GB is this useful or is the trade off of larger 
>  page tables and pages going to eat any benefit of being able to address 
>  these few bytes and if so when does the PAE kernel become useful?
>   
The answer is "it depends." If you have 4GB or more of usable RAM PAE 
will let you address all of it, and may increase you process max size 
depending on the options in your non-PAE kernel.

Look at /proc/mtrr to see what memory the OS thinks you have. If it 
doesn't agree with reality, use dmesg to look at the e820 registers, 
your BIOS may not set them correctly, the sizes should add up to 
physical memory, give or take memmapped hardware. If you have memort in 
mtrr ABOVE 4GB, PAE should help you.

[Roger Heflin]

Howard Wilkinson wrote:
>  Hmmm, I was afraid that this would be the answer! So the
> "definitive" 
>  answer is there is not one :-(
>  
>  Given that some of the BIOS comes from ad-on cards and this is going to 
>  affect each machine differently, then can the resulting map be different 
>  with different cards, or even with the same cards but in different 
>  slots, or even as in the bad old days of Windows 3.1 where the different 
>  value of the MAC address on a network card meant a different memory layout. 
Typically about the only cards that remap memory on a large scale (now) are 
video boards, I don't think that even the high-end IO (disk,infiniband) boards 
result in memory remapping, for them DMA to and from main memory is fast enough, 
for the video boards applications need high speed access to their memory so 
things can be displayed.   On my desktop board here if I have a nvidia 6200 
plugged in I lose 256MB (I don't even know if it is remapped to someplace I can 
even touch it if I were using a PAE kernel), without that board I have all of 
the ram, and I *only* have 3GB.

>  
>  The specific case that I have is 20 Intel SRMK4 machines all with 4GB of 
>  physical memory. Currently all running Fedora 32-bit kernels. I have yet 
>  to test the PAE kernels because I do not have any suitable benchmarks 
>  but I will see what happens. 
When I last tried a PAE vs non-PAE I could not see a big enough difference in 
speed to matter, I take it those machines aren't 64 bit capable?

How much memory does "cat /proc/meminfo" if you are getting within 256MB of 
4096M it is probably not worth messing with.

                              Roger
>  
>  Thanks to all for the responses.
>  
>  Howard.
>  
>  NOTE: my idea of useful is that the benefits outweigh the costs and the 
>  system gains overall. So a very complex calculation all round :-)
>

[Howard Wilkinson]

Roger Heflin wrote:
>  Howard Wilkinson wrote:
> > I am looking for a definitive answer to the question of where the PAE 
> > kernels become useful. I have seen various articles that mention 
> > needing PAE kernels if you have more then 4GB of physical memory in a 
> > 32-bit processor environment. I have also seen statements that say 
> > you need them if you have 4GB or more of memory. Now which is right? 
> > Also, even if you need a PAE kernel because the last few bytes are 
> > not addressable when you have exactly 4GB is this useful or is the 
> > trade off of larger page tables and pages going to eat any benefit of 
> > being able to address these few bytes and if so when does the PAE 
> > kernel become useful?
> >
> > Howard.
> >
>
>  It depends on the bios, you would have to try with and without the PAE 
>  kernel and see if the amount of usable ram changes.
>
>  Some bioses won't remap any memory below 4GB (that is covered by 
>  something else) to over 4GB, if your bios does not remap anything 
>  above 4GB when you only have 4GB (or less) then PAE won't buy you 
>  anything.    And since often moving the covered memory from below 4GB, 
>  sometimes means moving some non-covered memory and therefore lowering 
>  the memory usable for an OS that does not support PAE-ie that other 
>  OS, often the bios *WON'T* move the covered memory at all because it 
>  would lower the usable memory below 4GB for that other OS.
>
>  This is almost always true on the Desktop class MB's, and it is 
>  sometimes true on the higher end stuff also.
>
>                                Roger
>
>  
Hmmm, I was afraid that this would be the answer! So the "definitive" 
answer is there is not one :-(

Given that some of the BIOS comes from ad-on cards and this is going to 
affect each machine differently, then can the resulting map be different 
with different cards, or even with the same cards but in different 
slots, or even as in the bad old days of Windows 3.1 where the different 
value of the MAC address on a network card meant a different memory layout.

The specific case that I have is 20 Intel SRMK4 machines all with 4GB of 
physical memory. Currently all running Fedora 32-bit kernels. I have yet 
to test the PAE kernels because I do not have any suitable benchmarks 
but I will see what happens.

Thanks to all for the responses.

Howard.

NOTE: my idea of useful is that the benefits outweigh the costs and the 
system gains overall. So a very complex calculation all round :-)

[Roger Heflin]

Howard Wilkinson wrote:
>  I am looking for a definitive answer to the question of where the PAE
>
>  kernels become useful. I have seen various articles that mention needing 
>  PAE kernels if you have more then 4GB of physical memory in a 32-bit 
>  processor environment. I have also seen statements that say you need 
>  them if you have 4GB or more of memory. Now which is right? Also, even 
>  if you need a PAE kernel because the last few bytes are not addressable 
>  when you have exactly 4GB is this useful or is the trade off of larger 
>  page tables and pages going to eat any benefit of being able to address 
>  these few bytes and if so when does the PAE kernel become useful?
>  
>  Howard.
>   
It depends on the bios, you would have to try with and without the PAE kernel 
and see if the amount of usable ram changes.

Some bioses won't remap any memory below 4GB (that is covered by something else) 
to over 4GB, if your bios does not remap anything above 4GB when you only have 
4GB (or less) then PAE won't buy you anything.    And since often moving the 
covered memory from below 4GB, sometimes means moving some non-covered memory 
and therefore lowering the memory usable for an OS that does not support PAE-ie 
that other OS, often the bios *WON'T* move the covered memory at all because it 
would lower the usable memory below 4GB for that other OS.

This is almost always true on the Desktop class MB's, and it is sometimes true 
on the higher end stuff also.

                               Roger

[Patrick O'Callaghan]

On Thu, 2008-07-24 at 14:40 -0700, Nifty Fedora Mitch wrote:
>  On Thu, Jul 24, 2008 at 01:49:34PM +0100, Howard Wilkinson wrote:
>  > 
>  > I am looking for a definitive answer to the question of where the PAE  
>  > kernels become useful. I have seen various articles that mention needing  
>  > PAE kernels if you have more then 4GB of physical memory in a 32-bit  
>  > processor environment. I have also seen statements that say you need  
>  > them if you have 4GB or more of memory. Now which is right? Also, even  
>  > if you need a PAE kernel because the last few bytes are not addressable  
>  > when you have exactly 4GB is this useful or is the trade off of larger  
>  > page tables and pages going to eat any benefit of being able to address  
>  > these few bytes and if so when does the PAE kernel become useful?
>  >
>  
>  http://en.wikipedia.org/wiki/Physical_Address_Extension
>  
>  Can you be more specific and define 'useful'.
>  
>  In general on a 32 bit system you will have 32 bit pointers by default...
>  signed arithmetic gives you an effective 2GB process size.   Compare and
>  contrast lseek() and lseek64()... sizeof(off_t something). 
32 bits gives you a 4GB address space. Pointers are not signed (and the
off_t type reflects this, which is why it's different from int).

>  But if you have six 2GB processes running on a 6.x GB system is that
> useful?
>  Are you playing with one Big process.
>  
>  Do you have a test case  or pointer to a test case (best) so folks with
>  large memory systems can sanity test this for you? 
I suspect the question is related to accessing *physical* (not virtual)
memory.

poc

[Nifty Fedora Mitch]

On Thu, Jul 24, 2008 at 01:49:34PM +0100, Howard Wilkinson wrote:
>  
>  I am looking for a definitive answer to the question of where the PAE  
>  kernels become useful. I have seen various articles that mention needing  
>  PAE kernels if you have more then 4GB of physical memory in a 32-bit  
>  processor environment. I have also seen statements that say you need  
>  them if you have 4GB or more of memory. Now which is right? Also, even  
>  if you need a PAE kernel because the last few bytes are not addressable  
>  when you have exactly 4GB is this useful or is the trade off of larger  
>  page tables and pages going to eat any benefit of being able to address  
>  these few bytes and if so when does the PAE kernel become useful?
>  
http://en.wikipedia.org/wiki/Physical_Address_Extension

Can you be more specific and define 'useful'.

In general on a 32 bit system you will have 32 bit pointers by default...
signed arithmetic gives you an effective 2GB process size.   Compare and
contrast lseek() and lseek64()... sizeof(off_t something).

But if you have six 2GB processes running on a 6.x GB system is that useful?
Are you playing with one Big process.

Do you have a test case  or pointer to a test case (best) so folks with
large memory systems can sanity test this for you?


-- 
	T o m  M i t c h e l l 
	Looking for a place to hang my hat.