Hi
I wrote a very similar question some hours ago for Debian, which maybe got no replies becuase nobody is interested in running Debian on fedora.
The sad thing is, with the same problems and effects, I also don't get a Fedory system running that I instelled with yum. So, just replace debian with fedora 6 and debootstrap with yum in the mail below.
Again, This same system runs fine with Xen from xensource, but I'd like to use the Fedora Xen packages, for the security updates and stuff.
There must be something special in the Fedory Kernel that does make systems from "normal" partitions not work somehow, or something like that.
---------- Forwarded message ---------- From: Henning Sprang henning_sprang@gmx.de Date: Dec 18, 2006 1:27 AM Subject: Getting Debian domU running on fedora with fedora-native packages (works with xensource binaries) To: fedora-xen@redhat.com
Hi, I built a Debian domU on Fedora using debootstrap (which is really easy, when using alien to create a debootstrap rpm).
This VM runs very fine on fedora fc 6 with xen and the xen-linux kernel provides by xensource, and when built from sources.
When trying to use it with the fedora xen packages and kernels, I have absolutely no luck.
First, I was hanging on error message like "could not mount root device on /dev/root" (sorry, I don't have it at hand, becaue I just removed the fedora packages and used the xensource packages again, to be sure it really still works there).
I got past that by creating an initrd with --with=xennet --preload=xenblk options, and boot the domU with this instead ofthe initrd that comes with the fedora xen kernel package.
But then, the domU hangs on "starting crond..." I have no idea what is missing there, I already installed udev additionally, but that doesn't help.
As said, this same system runs with no problem on FC6 with the xensource binaries.
BTW: I run the system without pygrub and not with the kernel from disk, but with the kernel= options...
Henning
On Mon, Dec 18, 2006 at 04:36:18PM +0100, Henning Sprang wrote:
Hi
I wrote a very similar question some hours ago for Debian, which maybe got no replies becuase nobody is interested in running Debian on fedora.
The sad thing is, with the same problems and effects, I also don't get a Fedory system running that I instelled with yum. So, just replace debian with fedora 6 and debootstrap with yum in the mail below.
Again, This same system runs fine with Xen from xensource, but I'd like to use the Fedora Xen packages, for the security updates and stuff.
There must be something special in the Fedory Kernel that does make systems from "normal" partitions not work somehow, or something like that.
Most likely not got the correct set of modules built into the initrd. Upstream Xensource configs have most things built statically into the monolithic kernel. Fedora practice is to build everything as a module whenever possible, so we do this on Xen kernels, as well as baremetal. Or perhaps the Debian kernel utils aren't new enough to work on a 2.6.18 kernel ?
Finally, are there any SELinux AVC error messages in Dom0 logs ?
Regards, Dan.
Hi - thanks for the reply!
On 12/18/06, Daniel P. Berrange berrange@redhat.com wrote:
Most likely not got the correct set of modules built into the initrd. Upstream Xensource configs have most things built statically into the monolithic kernel.
Upstream Xen uses a lot of modules, but I rebuilt the sources without using modules. As that already worked, I did not yet try with the default xensource binaries - I gonna try that later and see if it also works, or if I just by accident built the only kernel allowing me to run debian on fedora.
Fedora practice is to build everything as a module whenever possible, so we do this on Xen kernels, as well as baremetal. Or perhaps the Debian kernel utils aren't new enough to work on a 2.6.18 kernel ?
Maybe, but sounds unlikely - when running on Debian, I use the debian Xen kernel version 2.6.17. but they also provide a 2.6.18 series kernel for non-xen and xen. So the tools for these kernek Versions must be there.
Finally, are there any SELinux AVC error messages in Dom0 logs ?
I don't see many selinux messages, I admit I have disabled selinux (I'm used to this from older fedora versions).
But what I see now is that the dom0 messages log is full of "4gb seg fixup, process ..." messages. On xen-users there is a hint on setting hwcap 0 segneg, but this is already done by a file in /etc/ld.so.conf, provided by the Fedora xen packages I guess.
I also tried further with a Fedora guest installed with yum, and this works in the meantime. Seems I probably did something wrong before, too hectic, because now it looks like making an own initrd with --with=xennet and --preload=xenblk seemed to help here. When this is running, also in the domU the logs and console are filled with the 4g seg fixup messages.
Still, with including these modules, Debian is hanging at boot after printing "Starting periodic command scheduler: crond."
Nothing happening anymore after this point.
Henning
On Mon, 18 Dec 2006 16:36:18 +0100 "Henning Sprang" henning_sprang@gmx.de wrote:
There must be something special in the Fedory Kernel that does make systems from "normal" partitions not work somehow, or something like that.
Not on my system at work. We installed a 32 bit hvm FC6 on a physical disk via virt-manager, and it worked fine under x86_64 dom0 (also FC6). I did manage to screw it up somehow trying to get the kernel updated, but it was working great before that :-).
Hi Tom,
On 12/18/06, Tom Horsley tomhorsley@adelphia.net wrote:
[...] Not on my system at work. We installed a 32 bit hvm FC6 on a physical disk via virt-manager, and it worked fine under x86_64 dom0 (also FC6). I did manage to screw it up somehow trying to get the kernel updated, but it was working great before that :-).
Seems I did not explain correctly what I meant.
With what you do, HVM Installs take place exactly not into partitions you defined from the outside, but into a block devices, and the hvm system then does full partitioning inside this.
What I mean what fedora kernels have problems with is when pointing the disk config directly onto a partition created from outside the domU.
An example:
In your setup you probably have a disk config line like this;
disk=[ 'phy:/dev/hdb1,hda,w']
What I mean is a setup that look like this:
disk = [ 'phy:/dev/hdb1,hda1,w']
where I create the ext3 filesystem from the dom0. These are mostly used with paravirtual systems, because on hvm you just use isos. But as I already said, these things can be handled by making a special initrd... Still, a Debian guest won't work, but works with no problem with a Kernel from xensource.
Henning
On Tue, 19 Dec 2006, Henning Sprang wrote:
as I already said, these things can be handled by making a special initrd... Still, a Debian guest won't work, but works with no problem with a Kernel from xensource.
There are many things that are unfortunately biting you when mixing debian and fedora kernels/xen. Unfortunate, because this is exactly what endusers of xen hope to archive.
I found the entire debian mkinitrd quite messy. they seem to no longer have mkinitrd, only as part of the yaird package, which is one that can only really generate an initrd image for the running kernel (yeah right, then i likely don't need it anymore eh?).
Using an initrd created with fedora on debian fails because they seem to be handling the change of root from initrd different. Might be related to pivot root changes?
On top of that, we ran into the pae vs no pae issue as well between these systems, so that we also could not boot fedora kernels/initrds from a debian xen0 either.
All in all, our migration from xen2 to xen3 has caused us many headaches, and so far, the xen3 setup is less flexable then we had before. I think if xen2 had been available for kernels > 2.6.12, I would have backed out halfway and gone back to xen2.
I did however, get some more appreciation for the additions that redhat has added to the xensource default builds. Perhaps I should phase out our < 512MB debian xen0's for some stripped down version of FC6 that fits in the same amount of space.
Paul
On Tue, Dec 19, 2006 at 12:06:42PM +0100, Henning Sprang wrote:
Hi Tom,
On 12/18/06, Tom Horsley tomhorsley@adelphia.net wrote:
[...] Not on my system at work. We installed a 32 bit hvm FC6 on a physical disk via virt-manager, and it worked fine under x86_64 dom0 (also FC6). I did manage to screw it up somehow trying to get the kernel updated, but it was working great before that :-).
Seems I did not explain correctly what I meant.
With what you do, HVM Installs take place exactly not into partitions you defined from the outside, but into a block devices, and the hvm system then does full partitioning inside this.
Paravirt installs should work in the same way - you are mapping through virtual disks, not virtual partitions. The latter just happens to work by chance, and is not supported.
What I mean what fedora kernels have problems with is when pointing the disk config directly onto a partition created from outside the domU.
Try mapping your disk through as 'xvda' instead of 'hda1'.
An example:
In your setup you probably have a disk config line like this;
disk=[ 'phy:/dev/hdb1,hda,w']
What I mean is a setup that look like this:
disk = [ 'phy:/dev/hdb1,hda1,w']
Two issues there
1. We strongly recommend *against* specifying hdX / sdX for virtual disk device type in paravirt guests. The Xen XVD driver is hijacking the IDE / SCSI drivers to do this hack. This will not be accepted upstream and so support will likely disappear in a future kernel-xen release. Use xvdX instead, eg
'phy:/dev/hdb1,xvda,w'
2. The first form of disk config you have is the preferred setup. ie map the partition from Dom0, through to DomU as a *disk*, not a partition. DomU should then partition this further as required, Your second config 'phy:/dev/hdb1,xvda1,w' may /happen/ to work, but there are no guarentees this will continue to work in the future. Only the 'phy:/dev/hdb1,xvda,w' format is supported for compatability long term.
eg, /dev/hdb1 in Dom0, maps to /dev/xvda in DomU then DomU can create /dev/xvda1 as needed
By having nested partition tables in this way, you ensure that Dom0 does not see the individual partitions from the DomU. If it were to see the indiivdual DomU partitions, and say both Dom0 and DomU have a partition formated ext3 with same label, then mount by label in Dom0 may end up picking up DomU's partition by accident. In addition tools in DomU may well get confuse by seeing 'xdva1' without a corresponding 'xvda'.
Regards, Dan.
-
Daniel P. Berrange -
Henning Sprang -
Paul Wouters -
Tom Horsley