cannot move Fedora Core 6 OS from one guest into another

[Adam Monsen]

I'm having trouble moving an entire Fedora Core 6 OS from one guest
(let's call it "OLD") into a new guest ("NEW"). I want to do this
because the old guest (which appears to be working just fine, thank
you) uses LVM, and I'd like to discontinue using LVM.

In case it saves you an hour or so of reading the rest of this email,
here's the error message:
-------------------8<-------------------
host# xm create -c /xen/new/conf
Using config file "./conf".
Going to boot Fedora Core (2.6.19-1.2895.fc6xen)
  kernel: /boot/vmlinuz-2.6.19-1.2895.fc6xen
  initrd: /boot/initrd-2.6.19-1.2895.fc6xen.img
Started domain manual3
Linux version 2.6.19-1.2895.fc6xen
(brewbuilder(a)hs20-bc2-2.build.redhat.com) (gcc version 4.1.1 20070105
(Red Hat 4.1.1-51)) #1 SMP Wed Jan 10 19:47:12 EST 2007
...
XENBUS: Device with no driver: device/vbd/51712
Freeing unused kernel memory: 184k freed
Write protecting the kernel read-only data: 387k
Red Hat nash version 5.1.19.0.2 starting
Mounting proc filesystem
Mounting sysfs filesystem
Creating /dev
Creating initial device nodes
Setting up hotplug.
Creating block device nodes.
Loading uhci-hcd.ko module
USB Universal Host Controller Interface driver v3.0
Loading ohci-hcd.ko module
Loading ehci-hcd.ko module
Loading jbd.ko module
Loading ext3.ko module
Loading xenblk.ko module
Registering block device major 202
 xvda: xvda1
Loading dm-mod.ko module
device-mapper: ioctl: 4.10.0-ioctl (2006-09-14) initialised: dm-devel(a)redhat.com
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Making device-mapper control node
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
  No volume groups found
Activating logical volumes
  Volume group "VolGroup00" not found
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!
------------------->8-------------------

I see nothing useful in /var/log/xend.log or /var/log/xend-debug.log,
just that the domain crashed and was destroyed.

Ok, now here's the whole story from the beginning...

OLD is a paravirtualized guest, NEW will also be. The host for both
OLD and NEW is an up-to-date FC6 system. The goal is to migrate OLD
into NEW, a created-from-scratch guest that doesn't use LVM to slice
up its file-backed VBD.

To try to make it easy on myself, I used qemu to boot a Fedora Core 6
rescue CD .iso image (hmm, might've be a FC5 rescue CD, but this
hopefully shouldn't matter based on what I did in rescue mode), as
follows:
-------------------8<-------------------
host# qemu -hda old.img -hdb new.img -cdrom FC-6-i386-rescuecd.iso -boot d
------------------->8-------------------

old.img is the file-backed VBD for OLD, and was created using a
GUI-guided network/anaconda/whatever install via the "Virtual Machine
Manager". The first partition is /boot, the second is an LVM volume
group with two logical volumes, LogVol00 being a 1.9G partition that
is mounted as the root partition, and LogVol01 is used for swap space.
-------------------8<-------------------
host# ls -lh old.img
-rw-r--r-- 1 root root 3.0G Feb  8 17:07 old.img

host# du -hs old.img
934M    old.img

host# fdisk -l old.img
last_lba(): I don't know how to handle files with mode 81a4
You must set cylinders.
You can do this from the extra functions menu.

Disk old.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
old.img1   *           1          13      104391   83  Linux
old.img2              14         382     2963992+  8e  Linux LVM
------------------->8-------------------

new.img was just a large sparse file, waiting to become a file-backed VBD.
-------------------8<-------------------
host# dd if=/dev/zero bs=1 count=1 seek=3G of=new.img
------------------->8-------------------

And qemu worked fine. In rescue mode, I was able to mount both
file-backed VBDs, partition/format hdb, copy over OLD to create NEW.
Seemed peachy.
-------------------8<-------------------
qemu# fdisk -l /dev/hdb

Disk /dev/hdb: 3221 MB, 3221225472 bytes
255 heads, 63 sectors/track, 391 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot   Start    End    Blocks     Id     System
/dev/hdb1            1    391   3140676     83     Linux

qemu# mkfs.ext3 /dev/hdb1
...worked fine...

qemu# mkdir /mnt/old
qemu# lvm vgchange -ay
  2 logical volume(s) in volume group "VolGroup00" now active
qemu# mount /dev/VolGroup00/LogVol00 /mnt/old
qemu# mount /dev/hda1 /mnt/old/boot
qemu# mkdir /mnt/new
qemu# mount /dev/hdb1 /mnt/new
qemu# cp -a /mnt/old/* /mnt/new
------------------->8-------------------

grub install onto hdb seemed to work fine, too:
-------------------8<-------------------
qemu# grub
grub> device (hd0) /dev/hdb
grub> root (hd0,0)
grub> setup (hd0)
...worked fine...
grub> quit
------------------->8-------------------

I then got out of rescue mode and quit qemu. I proceeded to modify NEW
slightly. First I mounted it):
-------------------8<-------------------
host# mkdir /mnt/new
host# bytes_sec=512 sect_track=63
host# mount -o loop,offset=$((bytes_sec * sect_track)) hd.img /mnt/new
------------------->8-------------------

I got that calculation for the offset from the Xen user manual. I was
somehow using this python script to derive the number of cylinders:
-------------------8<-------------------
host# cat disk_geom.py
size_GiB = 2.0
size_bytes = size_GiB * 1024 * 1024 * 1024
bytes_sec = 512
sec_track = 63
heads = 255
print "Cylinders: %d" % (size_bytes / (heads * sec_track * bytes_sec))
------------------->8-------------------

And yeah, I'm saying 2.0 GB instead of 3.0 GB in the script, and I'm
probably mixing up GiB and GB, and ...ugh. ANYWAY, the mount command
actually worked, so I guess 32256 (512 * 63) was correct. Or
something.

At any rate, I was able to modify files in the NEW's file-backed VBD.
/etc/grub.conf (a symlink to ../boot/grub/grub.conf) became:
-------------------8<-------------------
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Fedora Core (2.6.19-1.2895.fc6xen)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.19-1.2895.fc6xen ro root=/
        initrd /boot/initrd-2.6.19-1.2895.fc6xen.img
------------------->8-------------------

The Xen guest config for NEW looks like:
-------------------8<-------------------
name = 'new'
memory = '256'
disk = [ 'tap:aio:/var/xen/vm/new/hd.img,xvda,w' ]
vif = [ '' ]
vnc = 1
bootloader = '/usr/bin/pygrub'
on_reboot = 'restart'
on_crash = 'destroy'
------------------->8-------------------

NEW's /etc/fstab looks like:
-------------------8<-------------------
/dev/xvda1              /                       ext3    defaults        1 1
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
sysfs                   /sys                    sysfs   defaults        0 0
------------------->8-------------------

(nope, no swap, hope I don't need that).

I also changed NEW's /boot/grub/device.map, just in case:
-------------------8<-------------------
(hd0)   /dev/xvda
------------------->8-------------------

I then unmounted the file-backed VBD, started up NEW, and saw the
error I pasted at the top of this email.

Hints on how I could possibly make this any harder on myself would
also be appreciated. Thanks!

-- 
Adam Monsen