kernel-xen-guest-2.6.15-1.1955_FC5 seems to eat filesystems

[Steven Pritchard]

I updated my Xen test box to rawhide a few days ago, and I was happy
to find that it not only booted the hypervisor kernel (which hadn't
been working on this old dual PIII), but the guest kernels started up
too.

Now it looks like the filesystems on all three guests I was running
are trashed.  The first one I checked was so bad that e2fsck
segfaulted when I tried to repair it.

The host itself checks out fine, so it doesn't seem to be a problem
with kernel-xen-hypervisor.

Is this a known issue, or something I should bugzilla?

Oh, and these three guests were happily running for months on FC4...
Luckily they were disposable though.  :-)

Steve
-- 
Steven Pritchard - K&S Pritchard Enterprises, Inc.
Email: steve(a)kspei.com             http://www.kspei.com/
Phone: [phone]               Mobile: [phone]

[Stephen C. Tweedie]

Hi,

On Sat, 2006-02-18 at 20:30 -0600, Steven Pritchard wrote:
>  I updated my Xen test box to rawhide a few days ago, and I was happy
>  to find that it not only booted the hypervisor kernel (which hadn't
>  been working on this old dual PIII), but the guest kernels started up
>  too.
>  
>  Now it looks like the filesystems on all three guests I was running
>  are trashed.  The first one I checked was so bad that e2fsck
>  segfaulted when I tried to repair it.
>  
>  The host itself checks out fine, so it doesn't seem to be a problem
>  with kernel-xen-hypervisor.
>  
>  Is this a known issue, or something I should bugzilla? 
I haven't seen any such problems: indeed, a full guest install is a
pretty fs-intensive activity but that seems to work fine for me.  I last
tried with the 1.1948 kernels, though --- about to try with 1.1955 to
see how that goes.

Do you have any more details on what went wrong?

Thanks,
 Stephen

[Steven Pritchard]

On Mon, Feb 20, 2006 at 10:27:38AM -0500, Stephen C. Tweedie wrote:
>  Do you have any more details on what went wrong? 
I have logs for one of the guests:

Feb 18 21:44:32 mj2 kernel: inode_doinit_with_dentry:  context_to_sid(unlabeled) returned
22 for dev=dm-2 ino=196830
[repeats 56 times]
Feb 18 21:44:42 mj2 kernel: inode_doinit_with_dentry:  context_to_sid(unlabeled) returned
22 for dev=dm-2 ino=196948
Feb 18 21:48:09 mj2 kernel: end_request: I/O error, dev sda, sector 3614
Feb 18 21:48:09 mj2 kernel: Buffer I/O error on device dm-0, logical block 1615
Feb 18 21:48:09 mj2 kernel: lost page write due to I/O error on dm-0
Feb 18 21:48:09 mj2 kernel: Aborting journal on device dm-0.
Feb 18 21:48:09 mj2 kernel: ext3_abort called.
Feb 18 21:48:09 mj2 kernel: EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected
aborted journal
Feb 18 21:48:09 mj2 kernel: Remounting filesystem read-only

It was running ext3 on lvm2.

Steve
-- 
Steven Pritchard - K&S Pritchard Enterprises, Inc.
Email: steve(a)kspei.com             http://www.kspei.com/
Phone: [phone]               Mobile: [phone]

[Stephen C. Tweedie]

Hi,

On Mon, 2006-02-20 at 12:59 -0600, Steven Pritchard wrote:

>  Feb 18 21:44:32 mj2 kernel: inode_doinit_with_dentry: 
> context_to_sid(unlabeled) returned 22 for dev=dm-2 ino=196830
>  [repeats 56 times]
>  Feb 18 21:44:42 mj2 kernel: inode_doinit_with_dentry:  context_to_sid(unlabeled) returned
> 22 for dev=dm-2 ino=196948
>  Feb 18 21:48:09 mj2 kernel: end_request: I/O error, dev sda, sector 3614 
Is there any log on the dom0 corresponding to this error?

I suspect that this may simply be an artifact of the domU ABI changing
prior to the final release of Xen-3.0.0.  The upstream XenSource guest
OS ABI was still in flux prior to 3.0.0, so there's really no guarantee
that can be made to support running older kernels.  If you have the domU
running out of a chroot, can you use that to update its kernel to a
rawhide guest kernel/initrd and see if that helps?

--Stephen

[Steven Pritchard]

On Mon, Feb 20, 2006 at 02:53:24PM -0500, Stephen C. Tweedie wrote:
>  Is there any log on the dom0 corresponding to this error? 
None.

>  I suspect that this may simply be an artifact of the domU ABI
> changing
>  prior to the final release of Xen-3.0.0.  The upstream XenSource guest
>  OS ABI was still in flux prior to 3.0.0, so there's really no guarantee
>  that can be made to support running older kernels.  If you have the domU
>  running out of a chroot, can you use that to update its kernel to a
>  rawhide guest kernel/initrd and see if that helps? 
Already done.  I was running 2.6.15-1.1955_FC5hypervisor on the host,
and 2.6.15-1.1955_FC5guest for the guest.

BTW, it would be helpful if the Requires for the Xen guest kernels
were trimmed a bit so the newer kernels could be installed on older
(FC4, for example) guests.  Rather than install the kernel-xen-guest
rpm, I just used --with=xennet when I created the initrd so the guest
would have networking.  I can't think of any other modules not already
loaded in the initrd that would be absolutely required for the guest.

Steve
-- 
Steven Pritchard - K&S Pritchard Enterprises, Inc.
Email: steve(a)kspei.com             http://www.kspei.com/
Phone: [phone]               Mobile: [phone]

[Stephen C. Tweedie]

Hi,

On Mon, 2006-02-20 at 14:05 -0600, Steven Pritchard wrote:

>  BTW, it would be helpful if the Requires for the Xen guest kernels
>  were trimmed a bit so the newer kernels could be installed on older
>  (FC4, for example) guests. 
The requires: in there were added for a reason.  It's not hard to do an
rpm --nodeps to override, but we should be keeping the requires: by
default because there _are_ subtle (and sometimes not-so-subtle)
dependencies there.

--Stephen