64-Bit RAM Detection >4GB?

[Dustin Henning]

       I have a Core 2 Quad with 8 GiB of RAM.  When I installed FC7
(2.6.22.9-91.fc7), it only detected 4GiB (BIOS detects 8GiB).  I added
mem=8G to the kernel line in grub.conf, and then Gnome System Monitor showed
7.1GiB (vs I think 4.0).  I then changed that to mem=16G, and the Gnome
System Monitor shows 7.8 GiB.  /proc/meminfo MemTotal shows 8201832kB, so
that seems about right.  That said, the problem seems to be solved for the
kernel mentioned above.
	However, and this is why I am submitting this to the fedora-xen
mailing list, the virtualization kernel (2.6.20-2936.fc7xen) shows 3.1GiB in
Gnome System Monitor and 3214456KB in /proc/meminfo MemTotal.  Additionally,
"xm info" shows 3327 for total memory, so the problem isn't that it is
there
but not assigned to dom0.  I tried the same thing with grub.conf for xen,
but it did not work, mem=8G and mem=16G had no effect (this is shown to be a
valid parameter in the Xen manual at
[http://tx.downloads.xensource.com/downloads/docs/user/#SECTION0413000000000
0000000]).  Since it didn't work on the kernel line, I tried it on the
module (vmlinuz) line that has the other FC grub arguments, and that did not
work either; then I tried placing that on both lines (also to no avail).  I
also tried dom0_mem=8G on the kernel line, and that caused a kernel panic
stating not enough physical memory (got the same thing with dom0_mem=4G).
	Finally, as a side note, /proc/meminfo VmallocTotal shows
34359738367kB regardless of all of the above, and I don't know if that is
normal or not (swap partition is 2GiB).  Please let me know if you have
fedora 7 x86_64 xen working with >4GB of memory, if there is any additional
information that might be helpful, or if you have any suggestions I might
try in order to resolve this.  Thank you,
	Dustin

[Dale Bewley]

----- "Dustin Henning" <Dustin.Henning(a)prd-inc.com&gt; wrote:
>  I have a Core 2 Quad with 8 GiB of RAM.  When I installed FC7
>  (2.6.22.9-91.fc7), it only detected 4GiB (BIOS detects 8GiB).  I
>  added
>  mem=8G to the kernel line in grub.conf, and then Gnome System Monitor
>  showed
>  7.1GiB (vs I think 4.0).  I then changed that to mem=16G, and the
>  Gnome
>  System Monitor shows 7.8 GiB.  /proc/meminfo MemTotal shows 8201832kB,
>  so
>  that seems about right.  That said, the problem seems to be solved for
>  the
>  kernel mentioned above.
>  	However, and this is why I am submitting this to the fedora-xen
>  mailing list, the virtualization kernel (2.6.20-2936.fc7xen) shows
>  3.1GiB in
>  Gnome System Monitor and 3214456KB in /proc/meminfo MemTotal. 
>  Additionally,
>  "xm info" shows 3327 for total memory, so the problem isn't that it is
>  there
>  but not assigned to dom0.  I tried the same thing with grub.conf for
>  xen,
>  but it did not work, mem=8G and mem=16G had no effect (this is shown
>  to be a
>  valid parameter in the Xen manual at
>  [http://tx.downloads.xensource.com/downloads/docs/user/#SECTION0413000000000
>  0000000]).  Since it didn't work on the kernel line, I tried it on
>  the
>  module (vmlinuz) line that has the other FC grub arguments, and that
>  did not
>  work either; then I tried placing that on both lines (also to no
>  avail).  I
>  also tried dom0_mem=8G on the kernel line, and that caused a kernel
>  panic
>  stating not enough physical memory (got the same thing with
>  dom0_mem=4G).
>  	Finally, as a side note, /proc/meminfo VmallocTotal shows
>  34359738367kB regardless of all of the above, and I don't know if that
>  is
>  normal or not (swap partition is 2GiB).  Please let me know if you
>  have
>  fedora 7 x86_64 xen working with >4GB of memory, if there is any
>  additional
>  information that might be helpful, or if you have any suggestions I
>  might
>  try in order to resolve this.  Thank you,
>  	Dustin 
I have 4 procs with 16G RAM and no mem paramters in grub at all, and it looks like this:

Linux h 2.6.20-2931.fc7xen #1 SMP Mon Aug 13 10:11:56 EDT 2007 x86_64 x86_64 x86_64
GNU/Linux

# grep 'model name' /proc/cpuinfo
model name      : AMD Opteron(tm) Processor 852
model name      : AMD Opteron(tm) Processor 852
model name      : AMD Opteron(tm) Processor 852
model name      : AMD Opteron(tm) Processor 852

# grep Total /proc/meminfo
MemTotal:     13335552 kB
SwapTotal:     5261240 kB
VmallocTotal: 34359738367 kB

# xm info |grep total
total_memory           : 16319

I have 3 domU's running with each allocated 1G, so 16-3=13 which I suppose explains
the MemTotal line above from the perspective of dom0.


On an identical server running RHEL with no xen and I see something a bit different:

Red Hat Enterprise Linux AS release 4 (Nahant Update 5)
Linux z 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64
GNU/Linux

# grep Total /proc/meminfo
MemTotal:     16359284 kB
HighTotal:           0 kB
LowTotal:     16359284 kB
SwapTotal:     5261240 kB
VmallocTotal: 536870911 kB
HugePages_Total:     0

Not sure if that was helpful.
--
Dale Bewley - Unix Administrator - Shields Library - UC Davis
GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD  1753 064D 2583 B098 A0F3

[Dustin Henning]

> -----Original Message-----
> From: fedora-xen-bounces(a)redhat.com [mailto:[e-mail]] On Behalf Of
> Dale Bewley
> Sent: Friday, October 19, 2007 11:46
> To: fedora-xen
> Subject: Re: [Fedora-xen] 64-Bit RAM Detection >4GB?
>
> I have 4 procs with 16G RAM and no mem paramters in grub at all, and it looks like this:
>
> Linux h 2.6.20-2931.fc7xen #1 SMP Mon Aug 13 10:11:56 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>
> # grep 'model name' /proc/cpuinfo
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
>
> # grep Total /proc/meminfo
> MemTotal:     13335552 kB
> SwapTotal:     5261240 kB
> VmallocTotal: 34359738367 kB
>
> # xm info |grep total
> total_memory           : 16319
>
> I have 3 domU's running with each allocated 1G, so 16-3=13 which I suppose explains
> the MemTotal line above from the perspective of dom0.
>
>
> On an identical server running RHEL with no xen and I see something a bit different:
>
> Red Hat Enterprise Linux AS release 4 (Nahant Update 5)
> Linux z 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>
> # grep Total /proc/meminfo
> MemTotal:     16359284 kB
> HighTotal:           0 kB
> LowTotal:     16359284 kB
> SwapTotal:     5261240 kB
> VmallocTotal: 536870911 kB
> HugePages_Total:     0
>
> Not sure if that was helpful.
> --
> Dale Bewley - Unix Administrator - Shields Library - UC Davis
> GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD  1753 064D 2583 B098 A0F3 
Dale,
	That could be very helpful, or it might just get someone to blame Intel.  Either way,
thanks, now I at least know that it should work and I can try to determine why it
isn't.  As for the MemTotal, you are correct on why it is 13 instead of 16.  However,
if you shut down one of those domU's, the memory wouldn't necessarily be
automatically re-assigned to dom0, and /proc/meminfo is based on what is assigned to dom0.
 I think "xm info" would be the correct way to see the detected memory in the
system once domU's are created or if dom0 isn't assigned all memory.  There may be
other ways.  Thanks,
	Dustin

[Dustin Henning]

> -----Original Message-----
> From: fedora-xen-bounces(a)redhat.com [mailto:[e-mail]] On Behalf Of
> Dale Bewley
> Sent: Friday, October 19, 2007 11:46
> To: fedora-xen
> Subject: Re: [Fedora-xen] 64-Bit RAM Detection >4GB?
>
> I have 4 procs with 16G RAM and no mem paramters in grub at all, and it looks like this:
>
> Linux h 2.6.20-2931.fc7xen #1 SMP Mon Aug 13 10:11:56 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>
> # grep 'model name' /proc/cpuinfo
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
> model name      : AMD Opteron(tm) Processor 852
>
> # grep Total /proc/meminfo
> MemTotal:     13335552 kB
> SwapTotal:     5261240 kB
> VmallocTotal: 34359738367 kB
>
> # xm info |grep total
> total_memory           : 16319
>
> I have 3 domU's running with each allocated 1G, so 16-3=13 which I suppose explains
> the MemTotal line above from the perspective of dom0.
>
>
> On an identical server running RHEL with no xen and I see something a bit different:
>
> Red Hat Enterprise Linux AS release 4 (Nahant Update 5)
> Linux z 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>
> # grep Total /proc/meminfo
> MemTotal:     16359284 kB
> HighTotal:           0 kB
> LowTotal:     16359284 kB
> SwapTotal:     5261240 kB
> VmallocTotal: 536870911 kB
> HugePages_Total:     0
>
> Not sure if that was helpful.
> --
> Dale Bewley - Unix Administrator - Shields Library - UC Davis
> GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD  1753 064D 2583 B098 A0F3 
Dale,
	Could you tell me if you passed any arguments to the installer during your initial
install?  Also, can you show me the appropriate portion of your grub.conf?  I would like
to see if maybe it is different than mine even though they are both pre-packaged installs.
 I had a machine with 2.6.20-2931.fc7xen, so I copied that to the machine in question, and
it also only shows 3.1GiB for me.  I compared the config files for that kernel and the one
I was using, and they are identical (minus version and date comment lines), so that was as
I expected, but it makes it less likely that my problem is a version problem and more
likely that something else is haywire.  Thanks,
	Dustin

[Dustin Henning]

The grub installed with Fedora has a bug that causes xen (and sometimes
other kernels) to detect the memory wrong.
Bug URLs follow:
xen-users mailing list archive:
[url]http://lists.xensource.com/archives/html/xen-users/2007-08/msg00028.htm
l[/url]
Bugzilla:
[url]https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=250299[/url]

Theoretically, it could be argued that the BIOS is the problem and not grub,
I mean, it sounds like Dale's machine was functional without this patch, and
MSI isn't exactly a Linux Player.  Either way, the patch suggested in those
two locations resolved my issue.

-----Original Message-----
From: fedora-xen-bounces(a)redhat.com [mailto:[e-mail]]
On Behalf Of Dustin Henning
Sent: Thursday, October 18, 2007 16:00
To: fedora-xen(a)redhat.com
Subject: [Fedora-xen] 64-Bit RAM Detection >4GB?

       I have a Core 2 Quad with 8 GiB of RAM.  When I installed FC7
(2.6.22.9-91.fc7), it only detected 4GiB (BIOS detects 8GiB).  I added
mem=8G to the kernel line in grub.conf, and then Gnome System Monitor showed
7.1GiB (vs I think 4.0).  I then changed that to mem=16G, and the Gnome
System Monitor shows 7.8 GiB.  /proc/meminfo MemTotal shows 8201832kB, so
that seems about right.  That said, the problem seems to be solved for the
kernel mentioned above.
	However, and this is why I am submitting this to the fedora-xen
mailing list, the virtualization kernel (2.6.20-2936.fc7xen) shows 3.1GiB in
Gnome System Monitor and 3214456KB in /proc/meminfo MemTotal.  Additionally,
"xm info" shows 3327 for total memory, so the problem isn't that it is
there
but not assigned to dom0.  I tried the same thing with grub.conf for xen,
but it did not work, mem=8G and mem=16G had no effect (this is shown to be a
valid parameter in the Xen manual at
[http://tx.downloads.xensource.com/downloads/docs/user/#SECTION0413000000000
0000000]).  Since it didn't work on the kernel line, I tried it on the
module (vmlinuz) line that has the other FC grub arguments, and that did not
work either; then I tried placing that on both lines (also to no avail).  I
also tried dom0_mem=8G on the kernel line, and that caused a kernel panic
stating not enough physical memory (got the same thing with dom0_mem=4G).
	Finally, as a side note, /proc/meminfo VmallocTotal shows
34359738367kB regardless of all of the above, and I don't know if that is
normal or not (swap partition is 2GiB).  Please let me know if you have
fedora 7 x86_64 xen working with >4GB of memory, if there is any additional
information that might be helpful, or if you have any suggestions I might
try in order to resolve this.  Thank you,
	Dustin

[Richard W.M. Jones]

Dustin Henning wrote:
>  The grub installed with Fedora has a bug that causes xen (and
> sometimes
>  other kernels) to detect the memory wrong.
>  Bug URLs follow:
>  xen-users mailing list archive:
>  [url]http://lists.xensource.com/archives/html/xen-users/2007-08/msg00028.htm
>  l[/url]
>  Bugzilla:
>  [url]https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=250299[/url]
>  
>  Theoretically, it could be argued that the BIOS is the problem and not grub,
>  I mean, it sounds like Dale's machine was functional without this patch, and
>  MSI isn't exactly a Linux Player.  Either way, the patch suggested in those
>  two locations resolved my issue. 
Thanks for investigating this and posting the follow-up.  I did read 
your original report, but had no idea how to help.

Rich.

-- 
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom.  Registered in
England and Wales under Company Registration No. 03798903

[Dale Bewley]

On Fri, 2007-10-19 at 13:34 -0400, Dustin Henning wrote:
>  >-----Original Message-----
>  >From: fedora-xen-bounces(a)redhat.com [mailto:[e-mail]] On Behalf
> Of Dale Bewley
>  >Sent: Friday, October 19, 2007 11:46
>  >To: fedora-xen
>  >Subject: Re: [Fedora-xen] 64-Bit RAM Detection >4GB?
>  >
>  >I have 4 procs with 16G RAM and no mem paramters in grub at all, and it looks like
> this:
>  >
>  >Linux h 2.6.20-2931.fc7xen #1 SMP Mon Aug 13 10:11:56 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>  >
>  ># grep 'model name' /proc/cpuinfo
>  >model name      : AMD Opteron(tm) Processor 852
>  >model name      : AMD Opteron(tm) Processor 852
>  >model name      : AMD Opteron(tm) Processor 852
>  >model name      : AMD Opteron(tm) Processor 852
>  >
>  ># grep Total /proc/meminfo
>  >MemTotal:     13335552 kB
>  >SwapTotal:     5261240 kB
>  >VmallocTotal: 34359738367 kB
>  >
>  ># xm info |grep total
>  >total_memory           : 16319
>  >
>  >I have 3 domU's running with each allocated 1G, so 16-3=13 which I suppose
> explains the MemTotal line above from the perspective of dom0.
>  >
>  >
>  >On an identical server running RHEL with no xen and I see something a bit different:
>  >
>  >Red Hat Enterprise Linux AS release 4 (Nahant Update 5)
>  >Linux z 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64
> GNU/Linux
>  >
>  ># grep Total /proc/meminfo
>  >MemTotal:     16359284 kB
>  >HighTotal:           0 kB
>  >LowTotal:     16359284 kB
>  >SwapTotal:     5261240 kB
>  >VmallocTotal: 536870911 kB
>  >HugePages_Total:     0
>  >
>  >Not sure if that was helpful.
>  >--
>  >Dale Bewley - Unix Administrator - Shields Library - UC Davis
>  >GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD  1753 064D 2583 B098 A0F3
>  
>  Dale,
>  	Could you tell me if you passed any arguments to the installer during your initial
> install?  Also, can you show me the appropriate portion of your grub.conf?  I would like
> to see if maybe it is different than mine even though they are both pre-packaged installs.
>  I had a machine with 2.6.20-2931.fc7xen, so I copied that to the machine in question, and
> it also only shows 3.1GiB for me.  I compared the config files for that kernel and the one
> I was using, and they are identical (minus version and date comment lines), so that was as
> I expected, but it makes it less likely that my problem is a version problem and more
> likely that something else is haywire.  Thanks,
>  	Dustin 
My install command line was:
ksdevice=eth0 console=tty0 console=ttyS0,9600 load_ramdisk=1
initrd=f7/x86_64/initrd.img network ks=http://ks/ks-f7-v40z.cfg
kssendmac BOOT_IMAGE=f7/x86_64/vmlinuz

My grub entry for the currently running kernel is:
title Fedora (2.6.20-2931.fc7xen)
        root (hd0,0)
        kernel /xen.gz-2.6.20-2931.fc7 com1=9600
        module /vmlinuz-2.6.20-2931.fc7xen ro root=/dev/VGRAID/LVRoot
console=ttyS0,9600
        module /initrd-2.6.20-2931.fc7xen.img


My messy KS and KS post script is attached. 

During install, I did hit a bug
https://bugzilla.redhat.com/show_bug.cgi?id=242107 that I had to work
around by running mkinitrd before I rebooted. If I didn't I would be
missing the RAID1 module and fail to boot, but that's not related to the
memory issue.

-- 
Dale Bewley - Unix Administrator - Shields Library - UC Davis
GPG: 0xB098A0F3 0D5A 9AEB 43F4 F84C 7EFD  1753 064D 2583 B098 A0F3