Using large disk in DomU

[Robin Bowes]

Hi,

I'm running Xen 3.0.3 on FC6 with FC6 DomU guests.

On the Dom0 machine, I have 8 x 500GB disks running as a RAID6 array
with lvm on top of that. There is one partition available:
/dev/vg_media/lv_media which has an ext3 FS on it.

I want to make this available to a DomU guest, so I put this in the xen
config file:

disk = [ 'phy:/dev/vg_host/lv_slim,xvda,w',
         'phy:/dev/vg_media/lv_media,sda1,w', ]

The disk appears in the DomU guest, but if I fsck it, this is what I see:

# fsck /dev/sda1
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
The filesystem size (according to the superblock) is 732575744 blocks
The physical size of the device is 195704832 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

Any idea what's going on here? Am I doing something wrong?

R.

[Robin Bowes]

Robin Bowes wrote:
> Hi,
>
> I'm running Xen 3.0.3 on FC6 with FC6 DomU guests.
>
> On the Dom0 machine, I have 8 x 500GB disks running as a RAID6 array
> with lvm on top of that. There is one partition available:
> /dev/vg_media/lv_media which has an ext3 FS on it.
>
> I want to make this available to a DomU guest, so I put this in the xen
> config file:
>
> disk = [ 'phy:/dev/vg_host/lv_slim,xvda,w',
>          'phy:/dev/vg_media/lv_media,sda1,w', ]
>
> The disk appears in the DomU guest, but if I fsck it, this is what I see:
>
> # fsck /dev/sda1
> fsck 1.39 (29-May-2006)
> e2fsck 1.39 (29-May-2006)
> The filesystem size (according to the superblock) is 732575744 blocks
> The physical size of the device is 195704832 blocks
> Either the superblock or the partition table is likely to be corrupt!
> Abort<y>? yes
>
> Any idea what's going on here? Am I doing something wrong?

Robin Bowes wrote:
> Hi,
>
> I'm running Xen 3.0.3 on FC6 with FC6 DomU guests.
>
> On the Dom0 machine, I have 8 x 500GB disks running as a RAID6 array
> with lvm on top of that. There is one partition available:
> /dev/vg_media/lv_media which has an ext3 FS on it.
>
> I want to make this available to a DomU guest, so I put this in the xen
> config file:
>
> disk = [ 'phy:/dev/vg_host/lv_slim,xvda,w',
>          'phy:/dev/vg_media/lv_media,sda1,w', ]
>
> The disk appears in the DomU guest, but if I fsck it, this is what I see:
>
> # fsck /dev/sda1
> fsck 1.39 (29-May-2006)
> e2fsck 1.39 (29-May-2006)
> The filesystem size (according to the superblock) is 732575744 blocks
> The physical size of the device is 195704832 blocks
> Either the superblock or the partition table is likely to be corrupt!
> Abort<y>? yes
>
> Any idea what's going on here? Am I doing something wrong?

I tried a different approach to this.

I made /dev/md2 visible in the guest as xvdb1:

disk = [ 'phy:vg_host/lv_slim,xvda,w',
         'phy:md2,xvdb1,w', ]

Now, the VG is being detected when the guest boots up:

Scanning logical volumes
   Reading all physical volumes.  This may take a while...
   Found volume group "vg_media" using metadata type lvm2
   Found volume group "vg_slim" using metadata type lvm2

And, from a root shell, both the VG and the LV are visible:

# vgs
  VG       #PV #LV #SN Attr   VSize VFree
  vg_media   1   1   0 wz--n- 2.73T     0
  vg_slim    1   6   0 wz--n- 9.88G 32.00M

# lvdisplay vg_media/lv_media
  --- Logical volume ---
  LV Name                /dev/vg_media/lv_media
  VG Name                vg_media
  LV UUID                KK4J0k-OgoI-441I-lYa2-OF1k-C4FW-g4qi7o
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                2.73 TB
  Current LE             715406
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:6

/dev/mapper/vg_media-lv_media is present but there's no device file in
/dev, i.e. /dev/vg_media/lv_media does not exists.

I did notice an error if I try to activate all VGs:

# vgchange -ay
  device-mapper: reload ioctl failed: Invalid argument
  1 logical volume(s) in volume group "vg_media" now active
  6 logical volume(s) in volume group "vg_slim" now active

So, any idea what's going on here?

How can I make this filesystem available to my guest?

Thanks,

R.

[Robin Bowes]

Logged as a bug:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555

Also see this thread on xen-devel:

http://thread.gmane.org/gmane.comp.emulators.xen.devel/32640/focus=32640

This changeset has been applied to fix it:

http://xenbits.xensource.com/xen-unstable.hg?cs=0c0ef61de06b

R.

[Daniel P. Berrange]

On Tue, Nov 21, 2006 at 12:02:32PM +0000, Robin Bowes wrote:
> Logged as a bug:
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555
>
> Also see this thread on xen-devel:
>
> http://thread.gmane.org/gmane.comp.emulators.xen.devel/32640/focus=32640
>
> This changeset has been applied to fix it:
>
> http://xenbits.xensource.com/xen-unstable.hg?cs=0c0ef61de06b

If you want to build a new kernel RPM for testing this patch something along
the lines of the following should work.

1. Extract the patch file from mercurial

hg clone  http://xenbits.xensource.com/xen-unstable.hg
  cd xen-unstable.hg
  hg log --patch -r 0c0ef61de06b > ~/rpm/SOURCES/large-disks.patch

2. Grab the latest  kernel  src.rpm file & install it locally

rpm -ivh kernel-XXXXXX.src.rpm
     cd ~/rpm/SPECS

3. Edit the kenrel.spec file, incrementing release tag,

Release: %{release}.1xen2tbtest

Add

Patch30000: large-disks.patch

And in the %setup section

%patch30000 -p1

4. Rebuild the Xen kernel for i686

rpmbuild --target i686 --define 'buildup 0' --define 'buildsmp 0'
        --define 'buildkdump 0' -ba kernel-2.6.spec

..wait 30 minutes or so...

You should now have a shiny new kernel-xen RPM ready for install in Dom0/DomU

Regards,
Dan.

[Robin Bowes]

Daniel P. Berrange wrote:
> If you want to build a new kernel RPM for testing this patch something along
> the lines of the following should work.
> Extract the patch file from mercurial
>
> hg clone  http://xenbits.xensource.com/xen-unstable.hg
>   cd xen-unstable.hg
>   hg log --patch -r 0c0ef61de06b > ~/rpm/SOURCES/large-disks.patch
>
> Grab the latest  kernel  src.rpm file & install it locally
>
> rpm -ivh kernel-XXXXXX.src.rpm
>   cd ~/rpm/SPECS
>
> Edit the kenrel.spec file, incrementing release tag,
>
> Release: %{release}.1xen2tbtest
>
> Add
>  Patch30000: large-disks.patch
>
> And in the %setup section
>  %patch30000 -p1
>
> Rebuild the Xen kernel for i686
>
> rpmbuild --target i686 --define 'buildup 0' --define 'buildsmp 0'
>      --define 'buildkdump 0' -ba kernel-2.6.spec

I did this, but the patch failed:

Patch #30000 (large-disks.patch):
+ patch -p1 -s
The text leading up to this was:
--------------------------
|changeset:   12496:0c0ef61de06b
|user:        [e-mail]
|date:        Tue Nov 21 10:16:58 2006 +0000
|summary:     [BLK] Handle block devices with more than 2^32 sectors.
|
|diff -r 82ba3c086e6c -r 0c0ef61de06b
linux-2.6-xen-sparse/drivers/xen/blkback/common.h
|--- a/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
21 09:26:43 2006 +0000
|+++ b/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
21 10:16:58 2006 +0000
--------------------------
File to patch:

To be honest, it was creating the patch that I would struggle with - I'm
OK on modifying spec files,etc.

Any idea what's wrong?

R.

[Daniel P. Berrange]

On Tue, Nov 21, 2006 at 08:35:38PM +0000, Robin Bowes wrote:
> Daniel P. Berrange wrote:
> > If you want to build a new kernel RPM for testing this patch something along
> > the lines of the following should work.
> > Extract the patch file from mercurial
> >
> > hg clone  http://xenbits.xensource.com/xen-unstable.hg
> >   cd xen-unstable.hg
> >   hg log --patch -r 0c0ef61de06b > ~/rpm/SOURCES/large-disks.patch
> >
> > Grab the latest  kernel  src.rpm file & install it locally
> >
> > rpm -ivh kernel-XXXXXX.src.rpm
> >   cd ~/rpm/SPECS
> >
> > Edit the kenrel.spec file, incrementing release tag,
> >
> > Release: %{release}.1xen2tbtest
> >
> > Add
> >  Patch30000: large-disks.patch
> >
> > And in the %setup section
> >  %patch30000 -p1
> >
> > Rebuild the Xen kernel for i686
> >
> > rpmbuild --target i686 --define 'buildup 0' --define 'buildsmp 0'
> >      --define 'buildkdump 0' -ba kernel-2.6.spec
>
> I did this, but the patch failed:
>
> Patch #30000 (large-disks.patch):
> patch -p1 -s
> The text leading up to this was:
>
> |changeset:   12496:0c0ef61de06b
> |user:        [e-mail]
> |date:        Tue Nov 21 10:16:58 2006 +0000
> |summary:     [BLK] Handle block devices with more than 2^32 sectors.
> |
> |diff -r 82ba3c086e6c -r 0c0ef61de06b
> linux-2.6-xen-sparse/drivers/xen/blkback/common.h
> |--- a/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
> 21 09:26:43 2006 +0000
> |+++ b/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
> 21 10:16:58 2006 +0000
>
> File to patch:
>
> To be honest, it was creating the patch that I would struggle with - I'm
> OK on modifying spec files,etc.
>
> Any idea what's wrong?

Wrong patch stripping I think - it should actually be '-p2' instead 
of '-p1'.

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston.  [phone] -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|

[Robin Bowes]

Daniel P. Berrange wrote:
> On Tue, Nov 21, 2006 at 08:35:38PM +0000, Robin Bowes wrote:
> > I did this, but the patch failed:
> >
> > Patch #30000 (large-disks.patch):
> > patch -p1 -s
> > The text leading up to this was:
> >
> > |changeset:   12496:0c0ef61de06b
> > |user:        [e-mail]
> > |date:        Tue Nov 21 10:16:58 2006 +0000
> > |summary:     [BLK] Handle block devices with more than 2^32 sectors.
> > |
> > |diff -r 82ba3c086e6c -r 0c0ef61de06b
> > linux-2.6-xen-sparse/drivers/xen/blkback/common.h
> > |--- a/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
> > 21 09:26:43 2006 +0000
> > |+++ b/linux-2.6-xen-sparse/drivers/xen/blkback/common.h        Tue Nov
> > 21 10:16:58 2006 +0000
> >
> > File to patch:
> >
> > To be honest, it was creating the patch that I would struggle with - I'm
> > OK on modifying spec files,etc.
> >
> > Any idea what's wrong?
>
> Wrong patch stripping I think - it should actually be '-p2' instead 
> of '-p1'.

I tried -p2 - same problem.

R.

[Robin Bowes]

Robin Bowes wrote:
> Daniel P. Berrange wrote:
> > On Tue, Nov 21, 2006 at 08:35:38PM +0000, Robin Bowes wrote:
> > > Any idea what's wrong?
> >
> > Wrong patch stripping I think - it should actually be '-p2' instead 
> > of '-p1'.
>
> I tried -p2 - same problem.

Daniel,

I'm really struggling here - I can't seem to get the original patch to
apply, even manually.

I tried this:

Created ~/rpmbuild/SOURCES/xen-large-disk.patch using the content here:

http://xenbits.xensource.com/xen-unstable.hg?cs=0c0ef61de06b

Installed kernel-2.6.18-1.2849.fc6.src.rpm.

Ran "rpmbuild -bp ~/rpmbuild/SPECS/kernel-xen.spec"

Switched to ~/rpmbuild/BUILD/kernel-2.6.18

Ran patch -p2 < ../../SOURCES/xen-large-disks.patch

Result:

can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- a/linux-2.6-xen-sparse/drivers/xen/blkback/common.h Tue Nov 21
09:26:43 2006 +0000
|+++ b/linux-2.6-xen-sparse/drivers/xen/blkback/common.h Tue Nov 21
10:16:58 2006 +0000
--------------------------
File to patch:

Any idea what I'm doing wrong here?

R.

[Daniel P. Berrange]

On Tue, Nov 21, 2006 at 11:07:03PM +0000, Robin Bowes wrote:
> Robin Bowes wrote:
> > Daniel P. Berrange wrote:
> > > On Tue, Nov 21, 2006 at 08:35:38PM +0000, Robin Bowes wrote:
> > > > Any idea what's wrong?
> > >
> > > Wrong patch stripping I think - it should actually be '-p2' instead 
> > > of '-p1'.
> >
> > I tried -p2 - same problem.
>
> Daniel,
>
> I'm really struggling here - I can't seem to get the original patch to
> apply, even manually.

I've created a src.rpm from latest FC6 kernel including the neccessary
patch & uploaded it here:

http://people.redhat.com/berrange/kernel-2.6.18-1.2849.1dan2tb.src.rpm

You can just grab it and run

rpmbuild --rebuild --target i686  kernel-2.6.18-1.2849.1dan2tb.src.rpm

Due to disk space limitations I'll be removing it from this URL after a
day or two...

Regards,
Dan.

[Robin Bowes]

Daniel P. Berrange wrote:
> I've created a src.rpm from latest FC6 kernel including the neccessary
> patch & uploaded it here:
>
> http://people.redhat.com/berrange/kernel-2.6.18-1.2849.1dan2tb.src.rpm
>
> You can just grab it and run
>
> rpmbuild --rebuild --target i686  kernel-2.6.18-1.2849.1dan2tb.src.rpm
>
> Due to disk space limitations I'll be removing it from this URL after a
> day or two...

Dan,

Thanks for this.

I've grabbed it anyway, even though I've managed to get this to work.

It was just a matter of getting into the right directory:

cd ../linux-%{kversion}.%{_target_cpu}
%patch30000 -p2

The kernel rebuild is chugging away as I type.

Thanks again.

R.

[Robin Bowes]

I installed the patched kernel and xen RPMS this morning and can confirm
that this fixed my problem.

I can now see my 2.7TB partition correctly in the DomU guest.

Thanks to all for the quick fix, and to Dan for help with building the
modified kernel RPM.

R.

[Robin Bowes]

Robin Bowes wrote:
> I installed the patched kernel and xen RPMS this morning and can confirm
> that this fixed my problem.
>
> I can now see my 2.7TB partition correctly in the DomU guest.
>
> Thanks to all for the quick fix, and to Dan for help with building the
> modified kernel RPM.

Hi,

I applied this patch against kernel-xen-2.6.18-1.2849 and I see
kernel-xen-2.6.18-1.2868 is available in updates.

Can anyone tell me if the patch has made it into any release packages
yet so I can allow yum to upgrade my kernel?

Thanks,

R.