understanding applications and PAE

[Vanco, Don]

While I hate to use this as a support line I've gotten no help from the 2
local lugs I've queried, and know that there's some heavy hitters lurking
here.

I understand that the Linux kernel can (through PAE) provide memory paging
beyond 4GB of physical memory on 32-bit architecture.  What I need to
understand is: what are the requirements on the application side to utilize
this memory?  I'm looking for documentation / links that talk about
applications and PAE.

Case in point: Lotus Notes on OS400 can address huge amount of RAM, but when
it's running on Linux does it need to be compiled in a specific manner, or
with some specific bit of code / API to address more than the architecture
physically supports?

Thanx
Don

[Nathan Bryant]

I think you can't do that currently. You can have multiple processes 
that collectively use more than 4 gigs but each process can only address 
4 gigs.

Vanco, Don wrote:

> While I hate to use this as a support line I've gotten no help
> from the 2
> local lugs I've queried, and know that there's some heavy hitters lurking
> here.
>
> I understand that the Linux kernel can (through PAE) provide memory paging
> beyond 4GB of physical memory on 32-bit architecture.  What I need to
> understand is: what are the requirements on the application side to utilize
> this memory?  I'm looking for documentation / links that talk about
> applications and PAE.
>
> Case in point: Lotus Notes on OS400 can address huge amount of RAM, but when
> it's running on Linux does it need to be compiled in a specific manner, or
> with some specific bit of code / API to address more than the architecture
> physically supports?
>
> Thanx
> Don
>
>
> --
> fedora-test-list mailing list
> fedora-test-list(a)redhat.com
> http://www.redhat.com/mailman/listinfo/fedora-test-list
>
>   
>

[Alan Cox]

On Mon, Dec 08, 2003 at 03:41:17PM -0500, Vanco, Don wrote:
>  I understand that the Linux kernel can (through PAE) provide memory
> paging
>  beyond 4GB of physical memory on 32-bit architecture.  What I need to
>  understand is: what are the requirements on the application side to utilize
>  this memory?  I'm looking for documentation / links that talk about
>  applications and PAE. 
PAE is transparent. Unfortunately its a half-baked solution in some ways
because it only fixed physical address space limits. So you get 3-4Gb depending
on kernel and configuration virtual per process even if you have 64Gb physical.

You can do EMS like stuff where you allocate a set of shared memory
segments and switch them in or out as needed. Its ugly but for some
tasks works very well.

[James Olin Oden]

On Mon, 8 Dec 2003, Vanco, Don wrote:

>  While I hate to use this as a support line I've gotten no help
> from the 2
>  local lugs I've queried, and know that there's some heavy hitters lurking
>  here.
>  
>  I understand that the Linux kernel can (through PAE) provide memory paging
>  beyond 4GB of physical memory on 32-bit architecture.  What I need to
>  understand is: what are the requirements on the application side to utilize
>  this memory?  I'm looking for documentation / links that talk about
>  applications and PAE.
>  
>  Case in point: Lotus Notes on OS400 can address huge amount of RAM, but when
>  it's running on Linux does it need to be compiled in a specific manner, or
>  with some specific bit of code / API to address more than the architecture
>  physically supports?
>  
Hi Don,

We use the PAE feature where I work.  The deal is that this functionality 
only makes it possible for the kernel to access and use all this memory,
but each process still can only address 4Gig of memory less the amount of
memory under 4Gig the kernel is using.  If you wanted a single app
to use more than 4Gig, at this point you would need to create several
processes that share memory.  Even this this is very specific to the 
application as to whether such a scheme could be used effectively (you
would in effect be shuffling data around processes as needed).

Does anyone know of any other solutions than PAE?

Cheers...james 
>  Thanx
>  Don
>  
>  
>  --
>  fedora-test-list mailing list
>  fedora-test-list(a)redhat.com
>  http://www.redhat.com/mailman/listinfo/fedora-test-list
>

[Xose Vazquez Perez]

James Olin Oden wrote:

>  Does anyone know of any other solutions than PAE? 
it's called 64 bits

[James Olin Oden]

On Tue, 9 Dec 2003, Xose Vazquez Perez wrote:

>  James Olin Oden wrote:
>  
>  > Does anyone know of any other solutions than PAE?
>  
>  it's called 64 bits
>  
Toushee (or however that is spelled).  Point taken.

Cheers...james 
>  
>  --
>  fedora-test-list mailing list
>  fedora-test-list(a)redhat.com
>  http://www.redhat.com/mailman/listinfo/fedora-test-list
>

[Frederic Jean]

It's "touche".

Fred

On Mon, 2003-12-08 at 18:30, James Olin Oden wrote:

>  On Tue, 9 Dec 2003, Xose Vazquez Perez wrote:
>  
>  > James Olin Oden wrote:
>  > 
>  > > Does anyone know of any other solutions than PAE?
>  > 
>  > it's called 64 bits
>  >
>  Toushee (or however that is spelled).  Point taken.
>  
>  Cheers...james 
>  > 
>  > --
>  > fedora-test-list mailing list
>  > fedora-test-list(a)redhat.com
>  > http://www.redhat.com/mailman/listinfo/fedora-test-list
>  > 
>  
>  
>  --
>  fedora-test-list mailing list
>  fedora-test-list(a)redhat.com
>  http://www.redhat.com/mailman/listinfo/fedora-test-list

[James Olin Oden]

On Mon, 8 Dec 2003, Frederic Jean wrote:

>  It's "touche".
>  
Well I guess its my day to eat dirt (-;

Cheers...james 
>  Fred
>  
>  On Mon, 2003-12-08 at 18:30, James Olin Oden wrote:
>  
>  > On Tue, 9 Dec 2003, Xose Vazquez Perez wrote:
>  > 
>  > > James Olin Oden wrote:
>  > > 
>  > > > Does anyone know of any other solutions than PAE?
>  > > 
>  > > it's called 64 bits
>  > >
>  > Toushee (or however that is spelled).  Point taken.
>  > 
>  > Cheers...james 
>  > > 
>  > > --
>  > > fedora-test-list mailing list
>  > > fedora-test-list(a)redhat.com
>  > > http://www.redhat.com/mailman/listinfo/fedora-test-list
>  > > 
>  > 
>  > 
>  > --
>  > fedora-test-list mailing list
>  > fedora-test-list(a)redhat.com
>  > http://www.redhat.com/mailman/listinfo/fedora-test-list
>

[Alan Cox]

On Mon, Dec 08, 2003 at 07:16:48PM -0500, James Olin Oden wrote:
>  but each process still can only address 4Gig of memory less the
> amount of
>  memory under 4Gig the kernel is using.  If you wanted a single app
>  to use more than 4Gig, at this point you would need to create several
>  processes that share memory.  Even this this is very specific to the 
>  application as to whether such a scheme could be used effectively (you
>  would in effect be shuffling data around processes as needed).
>  
>  Does anyone know of any other solutions than PAE? 
A 64bit system. 

There really isnt a way to go beyond PAE type approaches without madness
and keep x86-32 behaviour. Intel didn't do PAE solely to be annoying, its
a real limitation and aspects of it hurt kernel performance too

[Chris Kloiber]

On Tue, 2003-12-09 at 08:16, James Olin Oden wrote:
>  On Mon, 8 Dec 2003, Vanco, Don wrote:
>  
>  > While I hate to use this as a support line I've gotten no help from the 2
>  > local lugs I've queried, and know that there's some heavy hitters lurking
>  > here.
>  > 
>  > I understand that the Linux kernel can (through PAE) provide memory paging
>  > beyond 4GB of physical memory on 32-bit architecture.  What I need to
>  > understand is: what are the requirements on the application side to utilize
>  > this memory?  I'm looking for documentation / links that talk about
>  > applications and PAE.
>  > 
>  > Case in point: Lotus Notes on OS400 can address huge amount of RAM, but when
>  > it's running on Linux does it need to be compiled in a specific manner, or
>  > with some specific bit of code / API to address more than the architecture
>  > physically supports?
>  >
>  Hi Don,
>  
>  We use the PAE feature where I work.  The deal is that this functionality 
>  only makes it possible for the kernel to access and use all this memory,
>  but each process still can only address 4Gig of memory less the amount of
>  memory under 4Gig the kernel is using.  If you wanted a single app
>  to use more than 4Gig, at this point you would need to create several
>  processes that share memory.  Even this this is very specific to the 
>  application as to whether such a scheme could be used effectively (you
>  would in effect be shuffling data around processes as needed).
>  
>  Does anyone know of any other solutions than PAE? 
Can you do some tricks like Oracle does? They can use very large shared
memory segments using /dev/shm I've seen customers use /dev/shm up to 6
gigs on a 8 Gig box, and 14 Gigs on a 16 Gig box. This is probably
slower than normal ram access, but faster than swapping to disk.
Remember I'm in Support, I can't help you program this.

-- 
Chris Kloiber
Red Hat, Inc.

[Nathan Bryant]

Sounds like you could treat /dev/shm as a large, linear file and mmap() 
one window into it at a time...

Chris Kloiber wrote:

> On Tue, 2003-12-09 at 08:16, James Olin Oden wrote:
>   
>
> >On Mon, 8 Dec 2003, Vanco, Don wrote:
> >
> >    
> >
> >>While I hate to use this as a support line I've gotten no help from the 2
> >>local lugs I've queried, and know that there's some heavy hitters lurking
> >>here.
> >>
> >>I understand that the Linux kernel can (through PAE) provide memory paging
> >>beyond 4GB of physical memory on 32-bit architecture.  What I need to
> >>understand is: what are the requirements on the application side to utilize
> >>this memory?  I'm looking for documentation / links that talk about
> >>applications and PAE.
> >>
> >>Case in point: Lotus Notes on OS400 can address huge amount of RAM, but when
> >>it's running on Linux does it need to be compiled in a specific manner, or
> >>with some specific bit of code / API to address more than the architecture
> >>physically supports?
> >>
> >>      
> >>
> >Hi Don,
> >
> >We use the PAE feature where I work.  The deal is that this functionality 
> >only makes it possible for the kernel to access and use all this memory,
> >but each process still can only address 4Gig of memory less the amount of
> >memory under 4Gig the kernel is using.  If you wanted a single app
> >to use more than 4Gig, at this point you would need to create several
> >processes that share memory.  Even this this is very specific to the 
> >application as to whether such a scheme could be used effectively (you
> >would in effect be shuffling data around processes as needed).
> >
> >Does anyone know of any other solutions than PAE?
> >    
> >
>
> Can you do some tricks like Oracle does? They can use very large shared
> memory segments using /dev/shm I've seen customers use /dev/shm up to 6
> gigs on a 8 Gig box, and 14 Gigs on a 16 Gig box. This is probably
> slower than normal ram access, but faster than swapping to disk.
> Remember I'm in Support, I can't help you program this.
>
>   
>

[Michael K. Johnson]

On Tue, Dec 09, 2003 at 12:00:38PM -0500, Nathan Bryant wrote:
>  Sounds like you could treat /dev/shm as a large, linear file and
> mmap() 
>  one window into it at a time... 
And in RHEL3 and 2.6 kernel, that's what remap_file_pages is for -- a fast
remapping that doesn't trash vmas.  Quoting from mm/fremap.c:

* this syscall works purely via pagetables, so it's the most efficient
* way to map the same (large) file into a given virtual window. Unlike
* mmap()/mremap() it does not create any new vmas. The new mappings are
* also safe across swapout.

michaelkjohnson

 "He that composes himself is wiser than he that composes a book."
 Linux Application Development                     -- Ben Franklin
 http://people.redhat.com/johnsonm/lad/