How to use virtio-scsi on F17 host and F17 guest

[Gianluca Cecchi]

Hello,
reading here:
https://fedoraproject.org/wiki/Features/virtio-scsi

it should be possible to use virtio-scsi, even if not completely
integrated in libvirt.
And installation would require this part for the hard disk:

-drive if=none,id=hd,file=/path/to/test.img -device
virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd

I have now an F17 host (kernel 3.3.7) and F17 guest (3.4.0) configured
with a virtio_blk disk
guest definition for the disk is this:
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/f17.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
    </disk>

Can I change it with "virsh edit" so that I can use virtio-scsi inside
guest and test?
What would be the xml snippet for it? Or do I have to use also a
wrapper to start qemu-kvm with particular options as during install?

Thanks in advance,
Gianluca

[Cole Robinson]

On 06/06/2012 12:30 PM, Gianluca Cecchi wrote:
> Hello,
> reading here:
> https://fedoraproject.org/wiki/Features/virtio-scsi
>
> it should be possible to use virtio-scsi, even if not completely
> integrated in libvirt.
> And installation would require this part for the hard disk:
>
> -drive if=none,id=hd,file=/path/to/test.img -device
> virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd
>
> I have now an F17 host (kernel 3.3.7) and F17 guest (3.4.0) configured
> with a virtio_blk disk
> guest definition for the disk is this:
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='qcow2'/>
>       <source file='/var/lib/libvirt/images/f17.img'/>
>       <target dev='vda' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>     </disk>
>
> Can I change it with "virsh edit" so that I can use virtio-scsi inside
> guest and test?
> What would be the xml snippet for it? Or do I have to use also a
> wrapper to start qemu-kvm with particular options as during install?

It looks like you want to add :

<controller type='scsi' index='0' model='virtio-scsi'/>

Then in change that <target line to

<target dev='sda' bus='scsi'/>

Problem is this will change the disk device node in the guest from vda to sda,
so could cause mounting problems.

- Cole

[Gianluca Cecchi]

On Thu, Jun 7, 2012 at 3:25 PM, Cole Robinson wrote:
> It looks like you want to add :
>
> <controller type='scsi' index='0' model='virtio-scsi'/>
>
> Then in change that <target line to
>
> <target dev='sda' bus='scsi'/>
>
> Problem is this will change the disk device node in the guest from vda to sda,
> so could cause mounting problems.
> Cole

Actually, at least in F17, both the kernel command line "root=..."
part and the device first column in fstab refer to UUID...
So I think it should be transparent for the guest os, as far as inside
the standard initrd there is the virtio_scsi.ko kernel module
available.
And it seems it is so:
$ gunzip < /boot/initramfs-3.3.7-1.fc17.x86_64.img | cpio -itdmv |
grep virtio_scsi
-rwxr--r--   1 root     root        16144 May 22 00:42
usr/lib/modules/3.3.7-1.fc17.x86_64/kernel/drivers/scsi/virtio_scsi.ko

current config
<disk type='file' device='disk'>
     <driver name='qemu' type='qcow2'/>
     <source file='/var/lib/libvirt/images/f17.img'/>
     <target dev='vda' bus='virtio'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
   </disk>

would then become
<disk type='file' device='disk'>
     <driver name='qemu' type='qcow2'/>
     <source file='/var/lib/libvirt/images/f17.img'/>
     <target dev='sda' bus='scsi'/>
   </disk>

What about
     <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
?
Cut out?

And after adding
<controller type='scsi' index='0' model='virtio-scsi'/>
can I cut out the
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
    </controller>

that I think is for current virtio_blk disk ?

Gianluca

[Cole Robinson]

On 06/07/2012 09:48 AM, Gianluca Cecchi wrote:
> On Thu, Jun 7, 2012 at 3:25 PM, Cole Robinson wrote:
> > It looks like you want to add :
> >
> > <controller type='scsi' index='0' model='virtio-scsi'/>
> >
> > Then in change that <target line to
> >
> > <target dev='sda' bus='scsi'/>
> >
> > Problem is this will change the disk device node in the guest from vda to sda,
> > so could cause mounting problems.
> > Cole
>
> Actually, at least in F17, both the kernel command line "root=..."
> part and the device first column in fstab refer to UUID...
> So I think it should be transparent for the guest os, as far as inside
> the standard initrd there is the virtio_scsi.ko kernel module
> available.
> And it seems it is so:
> $ gunzip < /boot/initramfs-3.3.7-1.fc17.x86_64.img | cpio -itdmv |
> grep virtio_scsi
> -rwxr--r--   1 root     root        16144 May 22 00:42
> usr/lib/modules/3.3.7-1.fc17.x86_64/kernel/drivers/scsi/virtio_scsi.ko
>
> current config
>
> <disk type='file' device='disk'>
>      <driver name='qemu' type='qcow2'/>
>      <source file='/var/lib/libvirt/images/f17.img'/>
>      <target dev='vda' bus='virtio'/>
>      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>    </disk>
>
> would then become
>
> <disk type='file' device='disk'>
>      <driver name='qemu' type='qcow2'/>
>      <source file='/var/lib/libvirt/images/f17.img'/>
>      <target dev='sda' bus='scsi'/>
>    </disk>
>
> What about
>      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
> ?
> Cut out?
>
> And after adding
>
> <controller type='scsi' index='0' model='virtio-scsi'/>
> can I cut out the
>     <controller type='virtio-serial' index='0'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
>     </controller>
>
> that I think is for current virtio_blk disk ?

Nope, that's something different, so keep that. But do drop the <address
type='pci'... line from the <disk> block, and libvirt will generate you a new one.

- Cole

[Gianluca Cecchi]

On Thu, Jun 7, 2012 at 4:01 PM, Cole Robinson wrote:
> Nope, that's something different, so keep that. But do drop the <address
> type='pci'... line from the <disk> block, and libvirt will generate you a new one.
> Cole

ah...ok sorry for the confusion.. too many virt* things ;-)

BTW it is working like a charm at the moment without any complaint.
Even the encrypted /home I have inside the guest that has
/dev/mapper/luks-4b0a873a-72bf-4153-b188-65ee9a6449f1
as first field of /etc/fstab
I'm going to do some testing.
Thank you very much in the mean time
Any problems with the disk config when managing any other parameter in
virt-manager for this vm, such as adding a network card or other
things?

For the note, after starting the domain, the <disk> and <controller>
blocks has been changed this way:

<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/f17.img'/>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

<controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
    </controller>

Gianluca

[Cole Robinson]

On 06/07/2012 10:33 AM, Gianluca Cecchi wrote:
> On Thu, Jun 7, 2012 at 4:01 PM, Cole Robinson wrote:
> > Nope, that's something different, so keep that. But do drop the <address
> > type='pci'... line from the <disk> block, and libvirt will generate you a new one.
> > Cole
>
> ah...ok sorry for the confusion.. too many virt* things ;-)
>
> BTW it is working like a charm at the moment without any complaint.
> Even the encrypted /home I have inside the guest that has
> /dev/mapper/luks-4b0a873a-72bf-4153-b188-65ee9a6449f1
> as first field of /etc/fstab
> I'm going to do some testing.
> Thank you very much in the mean time
> Any problems with the disk config when managing any other parameter in
> virt-manager for this vm, such as adding a network card or other
> things?
>
> For the note, after starting the domain, the <disk> and <controller>
> blocks has been changed this way:
> <disk type='file' device='disk'>
>   <driver name='qemu' type='qcow2'/>
>   <source file='/var/lib/libvirt/images/f17.img'/>
>   <target dev='sda' bus='scsi'/>
>   <address type='drive' controller='0' bus='0' target='0' unit='0'/>
> </disk>
>
> <controller type='scsi' index='0' model='virtio-scsi'>
>   <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>
> function='0x0'/>
>     </controller>
>
> Gianluca

That's all expected, it's libvirt adding stable device addressing so that some
future qemu update doesn't change defaults and rearrange your hardware, which
can upset guests (particularly windows).

- Cole

[Kashyap Chamarthy]

On 06/07/2012 06:55 PM, Cole Robinson wrote:
> On 06/06/2012 12:30 PM, Gianluca Cecchi wrote:
> > Hello,
> > reading here:
> > https://fedoraproject.org/wiki/Features/virtio-scsi
> >
> > it should be possible to use virtio-scsi, even if not completely
> > integrated in libvirt.
> > And installation would require this part for the hard disk:
> >
> > -drive if=none,id=hd,file=/path/to/test.img -device
> > virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd
> >
> > I have now an F17 host (kernel 3.3.7) and F17 guest (3.4.0) configured
> > with a virtio_blk disk
> > guest definition for the disk is this:
> >     <disk type='file' device='disk'>
> >       <driver name='qemu' type='qcow2'/>
> >       <source file='/var/lib/libvirt/images/f17.img'/>
> >       <target dev='vda' bus='virtio'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> > function='0x0'/>
> >     </disk>
> >
> > Can I change it with "virsh edit" so that I can use virtio-scsi inside
> > guest and test?
> > What would be the xml snippet for it? Or do I have to use also a
> > wrapper to start qemu-kvm with particular options as during install?
>
> It looks like you want to add :
>
> <controller type='scsi' index='0' model='virtio-scsi'/>
>
> Then in change that <target line to
>
> <target dev='sda' bus='scsi'/>
>
> Problem is this will change the disk device node in the guest from vda to sda,
> so could cause mounting problems.

Cole,

Just a slightly different question:

A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:

qemu-kvm invocation:
#---------------------------------------------------#
# /usr/bin/qemu-kvm \
 -device virtio-scsi-pci,id=scsi \
 -enable-kvm \
 -nographic -nodefconfig -nodefaults \
 -chardev stdio,id=charserial0 \
 -device isa-serial,chardev=charserial0,id=serial0 \
 -device sga \
 -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
 -initrd initramfs-3.2.7-1.fc16.x86_64.img \
 -append "console=ttyS0"
#---------------------------------------------------#
kashyap@~$ rpm -q qemu-kvm
qemu-kvm-1.0-17.fc16.x86_64
kashyap@~$
#---------------------------------------------------#

I get this Kernel Panic:
#---------------------------------------------------#
.
.
.

[    2.554591] Kernel panic - not syncing: Attempted to kill init!
[    2.556143] Pid: 1, comm: init Not tainted 3.2.7-1.fc16.x86_64 #1
[    2.557768] Call Trace:
[    2.558408]  [<ffffffff815d7510>] panic+0x91/0x1a7
[    2.559649]  [<ffffffff81072541>] do_exit+0x861/0x8a0
[    2.560925]  [<ffffffff810728d2>] do_group_exit+0x42/0xa0
[    2.562289]  [<ffffffff81072947>] sys_exit_group+0x17/0x20
[    2.563730]  [<ffffffff815e9d82>] system_call_fastpath+0x16/0x1b

#---------------------------------------------------#

Complete boot stdout:
http://kashyapc.fedorapeople.org/stdout-booting-kernel-qemu-kvm-virtio-scsi....

Any hints?
Though, I could try the way it was just discussed on this thread on F17. Still,just
curious, as the kernel versions I'm having does have all the required virtio-scsi bits.
> Cole
>
> virt mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/virt

[Kashyap Chamarthy]

On 06/07/2012 06:55 PM, Cole Robinson wrote:
> On 06/06/2012 12:30 PM, Gianluca Cecchi wrote:
> > Hello,
> > reading here:
> > https://fedoraproject.org/wiki/Features/virtio-scsi
> >
> > it should be possible to use virtio-scsi, even if not completely
> > integrated in libvirt.
> > And installation would require this part for the hard disk:
> >
> > -drive if=none,id=hd,file=/path/to/test.img -device
> > virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd
> >
> > I have now an F17 host (kernel 3.3.7) and F17 guest (3.4.0) configured
> > with a virtio_blk disk
> > guest definition for the disk is this:
> >     <disk type='file' device='disk'>
> >       <driver name='qemu' type='qcow2'/>
> >       <source file='/var/lib/libvirt/images/f17.img'/>
> >       <target dev='vda' bus='virtio'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> > function='0x0'/>
> >     </disk>
> >
> > Can I change it with "virsh edit" so that I can use virtio-scsi inside
> > guest and test?
> > What would be the xml snippet for it? Or do I have to use also a
> > wrapper to start qemu-kvm with particular options as during install?
>
> It looks like you want to add :
>
> <controller type='scsi' index='0' model='virtio-scsi'/>
>
> Then in change that <target line to
>
> <target dev='sda' bus='scsi'/>
>
> Problem is this will change the disk device node in the guest from vda to sda,
> so could cause mounting problems.

Cole,

A related question:

A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:

qemu-kvm invocation:
#---------------------------------------------------#
# /usr/bin/qemu-kvm \
 -device virtio-scsi-pci,id=scsi \
 -enable-kvm \
 -nographic -nodefconfig -nodefaults \
 -chardev stdio,id=charserial0 \
 -device isa-serial,chardev=charserial0,id=serial0 \
 -device sga \
 -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
 -initrd initramfs-3.2.7-1.fc16.x86_64.img \
 -append "console=ttyS0"
#---------------------------------------------------#
kashyap@~$ rpm -q qemu-kvm
qemu-kvm-1.0-17.fc16.x86_64
kashyap@~$
#---------------------------------------------------#

I get this Kernel Panic:
#---------------------------------------------------#
.
.
.

[    2.554591] Kernel panic - not syncing: Attempted to kill init!
[    2.556143] Pid: 1, comm: init Not tainted 3.2.7-1.fc16.x86_64 #1
[    2.557768] Call Trace:
[    2.558408]  [<ffffffff815d7510>] panic+0x91/0x1a7
[    2.559649]  [<ffffffff81072541>] do_exit+0x861/0x8a0
[    2.560925]  [<ffffffff810728d2>] do_group_exit+0x42/0xa0
[    2.562289]  [<ffffffff81072947>] sys_exit_group+0x17/0x20
[    2.563730]  [<ffffffff815e9d82>] system_call_fastpath+0x16/0x1b

#---------------------------------------------------#

Complete boot stdout:
http://kashyapc.fedorapeople.org/stdout-booting-kernel-qemu-kvm-virtio-scsi....

Any hints?
Though, I could try the way it was just discussed on this thread on F17. Still,just
curious, as the kernel versions I'm having does have all the required virtio-scsi bits.
> Cole
>
> virt mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/virt

[Kevin Wolf]

Am 07.06.2012 19:50, schrieb Kashyap Chamarthy:
> A related question:
>
> A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:
>
> qemu-kvm invocation:
> #---------------------------------------------------#
> # /usr/bin/qemu-kvm \
>  -device virtio-scsi-pci,id=scsi \
>  -enable-kvm \
>  -nographic -nodefconfig -nodefaults \
>  -chardev stdio,id=charserial0 \
>  -device isa-serial,chardev=charserial0,id=serial0 \
>  -device sga \
>  -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
>  -initrd initramfs-3.2.7-1.fc16.x86_64.img \
>  -append "console=ttyS0"
> #---------------------------------------------------#

You're only adding the virtio-scsi controller here, but no disks, so
your guest can't find its root file system. I think you need to add:

-drive file=disk.img,if=none,id=sda
-device scsi-disk,bus=scsi.0,drive=sda

Kevin

[Kashyap Chamarthy]

(Sorry for the double email, previously. I thought it didn't went through, and re-sent it.)

On 06/08/2012 03:02 PM, Kevin Wolf wrote:
> Am 07.06.2012 19:50, schrieb Kashyap Chamarthy:
> > A related question:
> >
> > A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:
> >
> > qemu-kvm invocation:
> > #---------------------------------------------------#
> > # /usr/bin/qemu-kvm \
> >  -device virtio-scsi-pci,id=scsi \
> >  -enable-kvm \
> >  -nographic -nodefconfig -nodefaults \
> >  -chardev stdio,id=charserial0 \
> >  -device isa-serial,chardev=charserial0,id=serial0 \
> >  -device sga \
> >  -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
> >  -initrd initramfs-3.2.7-1.fc16.x86_64.img \
> >  -append "console=ttyS0"
> > #---------------------------------------------------#
>
> You're only adding the virtio-scsi controller here, but no disks, so
> your guest can't find its root file system. I think you need to add:
>
> -drive file=disk.img,if=none,id=sda
> -device scsi-disk,bus=scsi.0,drive=sda

Um, with passing the disk image(which has f16) explicitly, it still complaints can't find
the root= argument.

#--------------------------------------------------#
/usr/bin/qemu-kvm \
 -device virtio-scsi-pci,id=scsi \
 -enable-kvm \
 -nographic -nodefconfig -nodefaults \
 -chardev stdio,id=charserial0 \
 -device isa-serial,chardev=charserial0,id=serial0 \
 -drive file=/var/lib/libvirt/images/f16t1.qcow2,if=none,id=sda \
 -device scsi-disk,bus=scsi.0,drive=sda \
 -device sga \
 -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
 -initrd initramfs-3.2.7-1.fc16.x86_64.img \
 -append "console=ttyS0"
#--------------------------------------------------#
kashyap@~$ ps -ef | grep qemu-kvm | grep virtio-scsi
root     12705 12704 99 17:49 pts/11   00:00:04 /usr/bin/qemu-kvm -device
virtio-scsi-pci,id=scsi -enable-kvm -nographic -nodefconfig -nodefaults -chardev
stdio,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -drive
file=/var/lib/libvirt/images/f16t1.qcow2,if=none,id=sda -device
scsi-disk,bus=scsi.0,drive=sda -device sga -kernel vmlinuz-3.2.7-1.fc16.x86_64 -initrd
initramfs-3.2.7-1.fc16.x86_64.img -append console=ttyS0
kashyap@~$
#--------------------------------------------------#

Actually, I've stumbled across this bug above w/ virt-install while initially testing
F17(now resolved, though) -- https://bugzilla.redhat.com/show_bug.cgi?id=785815

#--------------------------------------------------#
.
.
.
[    1.972021] dracut Warning: dracut: FATAL: No or empty root= argument
dracut Warning: dracut: FATAL: No or empty root= argument
[    1.973816] dracut Warning: dracut: Refusing to continue
dracut Warning: dracut: Refusing to continue

[    1.975691] Kernel panic - not syncing: Attempted to kill init!
[    1.976631] Pid: 1, comm: init Not tainted 3.2.7-1.fc16.x86_64 #1
[    1.977596] Call Trace:
[    1.977998]  [<ffffffff815d7510>] panic+0x91/0x1a7
[    1.978853]  [<ffffffff81072541>] do_exit+0x861/0x8a0
[    1.979650]  [<ffffffff810728d2>] do_group_exit+0x42/0xa0
[    1.980495]  [<ffffffff81072947>] sys_exit_group+0x17/0x20
[    1.981354]  [<ffffffff815e9d82>] system_call_fastpath+0x16/0x1b
#--------------------------------------------------#
> Kevin

[Kevin Wolf]

Am 08.06.2012 14:25, schrieb Kashyap Chamarthy:
> (Sorry for the double email, previously. I thought it didn't went through, and re-sent it.)
>
> On 06/08/2012 03:02 PM, Kevin Wolf wrote:
> > Am 07.06.2012 19:50, schrieb Kashyap Chamarthy:
> > > A related question:
> > >
> > > A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:
> > >
> > > qemu-kvm invocation:
> > > #---------------------------------------------------#
> > > # /usr/bin/qemu-kvm \
> > >  -device virtio-scsi-pci,id=scsi \
> > >  -enable-kvm \
> > >  -nographic -nodefconfig -nodefaults \
> > >  -chardev stdio,id=charserial0 \
> > >  -device isa-serial,chardev=charserial0,id=serial0 \
> > >  -device sga \
> > >  -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
> > >  -initrd initramfs-3.2.7-1.fc16.x86_64.img \
> > >  -append "console=ttyS0"
> > > #---------------------------------------------------#
> >
> > You're only adding the virtio-scsi controller here, but no disks, so
> > your guest can't find its root file system. I think you need to add:
> >
> > -drive file=disk.img,if=none,id=sda
> > -device scsi-disk,bus=scsi.0,drive=sda
>
> Um, with passing the disk image(which has f16) explicitly, it still complaints can't find
> the root= argument.

Well, maybe you need to pass the root=... parameter in -append then. :-)

What it needs to be exactly depends on your setup. The GRUB
configuration on the guest should tell you the right value.

Or am I missing some additional requirement, Paolo?

Kevin

[Paolo Bonzini]

Il 08/06/2012 14:51, Kevin Wolf ha scritto:
> Am 08.06.2012 14:25, schrieb Kashyap Chamarthy:
> > (Sorry for the double email, previously. I thought it didn't went through, and re-sent it.)
> >
> > On 06/08/2012 03:02 PM, Kevin Wolf wrote:
> > > Am 07.06.2012 19:50, schrieb Kashyap Chamarthy:
> > > > A related question:
> > > >
> > > > A while ago, I was using just bare qemu-kvm to invoke an f16 kernel to test virtio-scsi:
> > > >
> > > > qemu-kvm invocation:
> > > > #---------------------------------------------------#
> > > > # /usr/bin/qemu-kvm \
> > > >  -device virtio-scsi-pci,id=scsi \
> > > >  -enable-kvm \
> > > >  -nographic -nodefconfig -nodefaults \
> > > >  -chardev stdio,id=charserial0 \
> > > >  -device isa-serial,chardev=charserial0,id=serial0 \
> > > >  -device sga \
> > > >  -kernel vmlinuz-3.2.7-1.fc16.x86_64 \
> > > >  -initrd initramfs-3.2.7-1.fc16.x86_64.img \
> > > >  -append "console=ttyS0"
> > > > #---------------------------------------------------#
> > >
> > > You're only adding the virtio-scsi controller here, but no disks, so
> > > your guest can't find its root file system. I think you need to add:
> > >
> > > -drive file=disk.img,if=none,id=sda
> > > -device scsi-disk,bus=scsi.0,drive=sda
> >
> > Um, with passing the disk image(which has f16) explicitly, it still complaints can't find
> > the root= argument.
>
> Well, maybe you need to pass the root=... parameter in -append then. :-)
>
> What it needs to be exactly depends on your setup. The GRUB
> configuration on the guest should tell you the right value.
>
> Or am I missing some additional requirement, Paolo?

No, I think you're right.  If the kernel command-line uses UUIDs for the
root image, the same image can usually switch transparently among
IDE/virtio-scsi/virtio-blk.

Paolo

[Gianluca Cecchi]

One of the main reasons for passing from virtio_blk to virtio_scsi
should be hot plug and hot resize (I hoped).
But with f17 host and f17 guest it seems not working, at least for me...

On host
$ uname -r
3.4.0-1.fc17.x86_64

qemu-kvm command line
$ ps -ef|grep f17
qemu     17838     1 24 18:37 ?        00:03:57 /usr/bin/qemu-kvm -S
-M pc-0.14 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1
-name f17 -uuid 259201b8-0699-6213-c583-e85bd3a05f1e -nodefconfig
-nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/f17.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime
-no-shutdown -device
virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 -drive
file=/var/lib/libvirt/images/f17.img,if=none,id=drive-scsi0-0-0-0,format=qcow2
-device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
-drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:b0:6c:5f,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-spice port=5900,addr=127.0.0.1,disable-ticketing -vga qxl -global
qxl-vga.vram_size=67108864 -device
intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

$ sudo qemu-img info /var/lib/libvirt/images/f17.img
image: /var/lib/libvirt/images/f17.img
file format: qcow2
virtual size: 7.0G (7516192768 bytes)
disk size: 5.0G
cluster_size: 65536

In guest:
# fdisk -l /dev/sda

Disk /dev/sda: 7516 MB, 7516192768 bytes
232 heads, 62 sectors/track, 1020 cylinders, total 14680064 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00031461

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    14680063     6827008   8e  Linux LVM

On host
$ sudo qemu-img resize /var/lib/libvirt/images/f17.img +1G
Image resized.

$ sudo qemu-img info /var/lib/libvirt/images/f17.img
[sudo] password for g.cecchi:
image: /var/lib/libvirt/images/f17.img
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 5.0G
cluster_size: 65536

But in f17 guest
# uname -r
3.4.0-1.fc17.x86_64

# cat /sys/class/scsi_device/2:0:0:0/device/model
QEMU HARDDISK

Tried these sort of commands to rescan

# echo "- - -" > /sys/class/scsi_host/host2/scan
(expected to work only for new added disk, not resize of pre-existing)

# echo "1" > /sys/class/scsi_device/2:0:0:0/device/rescan

# echo 1 > /sys/block/sda/device/rescan

# echo 1 > /sys/bus/scsi/devices/2:0:0:0/rescan

# blockdev --rereadpt /dev/sda
BLKRRPART: Device or resource busy

# partprobe -s /dev/sda
/dev/sda: msdos partitions 1 2

But no change in guest ....
Any parameter to set for guest or kernel module to load in guest to
allow this dynamic change?
At this moment in guest:

[root@f17 g.cecchi]# lsmod|grep hot
[root@f17 g.cecchi]# lsmod|grep plug
[root@f17 g.cecchi]# lsmod|grep acpi
[root@f17 g.cecchi]#

Thanks,
Gianluca

[Kevin Wolf]

Am 12.06.2012 19:04, schrieb Gianluca Cecchi:
> One of the main reasons for passing from virtio_blk to virtio_scsi
> should be hot plug and hot resize (I hoped).
> But with f17 host and f17 guest it seems not working, at least for me...
> In guest:
> # fdisk -l /dev/sda
>
> Disk /dev/sda: 7516 MB, 7516192768 bytes
> 232 heads, 62 sectors/track, 1020 cylinders, total 14680064 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0x00031461
>
> Device Boot      Start         End      Blocks   Id  System
> /dev/sda1   *        2048     1026047      512000   83  Linux
> /dev/sda2         1026048    14680063     6827008   8e  Linux LVM
>
> On host
> $ sudo qemu-img resize /var/lib/libvirt/images/f17.img +1G
> Image resized.

Never use qemu-img to modify an image that is in use by a VM, it might
break the image. You need to use the qemu monitor of that VM to modify
this image, which in your case means that you should probably use the
proper libvirt interface, e.g. with virsh.

Kevin

[Gianluca Cecchi]

On Wed, Jun 13, 2012 at 10:22 AM, Kevin Wolf wrote:
> Never use qemu-img to modify an image that is in use by a VM, it might
> break the image. You need to use the qemu monitor of that VM to modify
> this image, which in your case means that you should probably use the
> proper libvirt interface, e.g. with virsh.
>
> Kevin

ops .. I didn't keep this into consideration, as I only used qemu-img
resize with powered off vm until now...
So the disk was indeed brutally corrupted at next reboot... never mind.
I reinstalled f17 guest from scratch.
Now we are here:

f17 vm is running

on host:
$ sudo qemu-img info /var/lib/libvirt/images/f17.img
image: /var/lib/libvirt/images/f17.img
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 5.7G
cluster_size: 65536

[g.cecchi@ope46 ~]$ sudo virsh blockresize f17
/var/lib/libvirt/images/f17.img 9G
Block device '/var/lib/libvirt/images/f17.img' is resized

In guest the / filesystem is on lvm while /boot is on first partition:
[root@f17 ~]# fdisk -l /dev/sda

Disk /dev/sda: 8589 MB, 8589934592 bytes
64 heads, 32 sectors/track, 8192 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0001e726

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    16777215     7875584   8e  Linux LVM

[root@f17 ~]# echo 1 > /sys/block/sda/device/rescan

from dmesg output:
[  263.003897] sd 2:0:0:0: [sda] 18874368 512-byte logical blocks:
(9.66 GB/9.00 GiB)
[  263.004344] sda: detected capacity change from 8589934592 to 9663676416

[root@f17 ~]# fdisk -l /dev/sda

Disk /dev/sda: 9663 MB, 9663676416 bytes
64 heads, 32 sectors/track, 9216 cylinders, total 18874368 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0001e726

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    16777215     7875584   8e  Linux LVM

I run fdisk and delete / create lvm partition bigger (as I usually do
with plain partitions for filesystems on them); then

[root@f17 ~]# partprobe /dev/sda
Error: Partition(s) 2 on /dev/sda have been written, but we have been
unable to inform the kernel of the change, probably because it/they
are in use.  As a result, the old partition(s) will remain in use.
You should reboot now before making further changes.

[root@f17 ~]# kpartx /dev/sda
sda1 : 0 1024000 /dev/sda 2048
sda2 : 0 17848320 /dev/sda 1026048

[root@f17 ~]# pvscan
  PV /dev/sda2   VG vg_f17   lvm2 [7.50 GiB / 0    free]
  Total: 1 [7.50 GiB] / in use: 1 [7.50 GiB] / in no VG: 0 [0   ]

[root@f17 ~]# pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

but nothing changes actually...

[root@f17 ~]# pvs
  PV         VG     Fmt  Attr PSize PFree
  /dev/sda2  vg_f17 lvm2 a--  7.50g    0

Is this a limitation of lvm layer?
I noticed this kind of limitation also on VMware guests (with lsi
logic virtual controller) where apparently I'm unable to inform lvm of
the PV changed size....
I know that I can add another partition at the end of disk instead of
resizing the existing one... but I'd prefer this latter...
Thanks,
Gianluca

[Kevin Wolf]

Am 14.06.2012 21:55, schrieb Gianluca Cecchi:
> [root@f17 ~]# echo 1 > /sys/block/sda/device/rescan
>
> from dmesg output:
> [  263.003897] sd 2:0:0:0: [sda] 18874368 512-byte logical blocks:
> (9.66 GB/9.00 GiB)
> [  263.004344] sda: detected capacity change from 8589934592 to 9663676416

So at this point you have resized the image and told the kernel about
it, you're done with the virt-specific part. The rest should be the same
as on physical hardware.

A quick search on the net suggests indeed that you can't modify a
partition that is mounted/in use by LVM. So I'm afraid a new partition
that is added to the VG must be good enough.

Kevin

[Gianluca Cecchi]

On Fri, Jun 15, 2012 at 10:20 AM, Kevin Wolf  wrote:
> A quick search on the net suggests indeed that you can't modify a
> partition that is mounted/in use by LVM. So I'm afraid a new partition
> that is added to the VG must be good enough.
>
> Kevin

Actually the trick is to create PV on plain disks and not on partitions.
(thanks to the link
http://v-reality.info/2010/06/working-with-linux-volumes-n-vsphere/ I
found)
So a flexible strategy could be to make a system with
- first disk of 500Mb containing only one primary partition for /boot
(or evenually bigger to accomdate another partition for swap if one
prefers it over lvm based swap)
- second disk configured as PV for all other file systems

Successfully tested (actually on second disk)
On host
- create 4Gb disk
qemu-img create -f qcow2 /var/lib/libvirt/images/f17_sdb.img 4G
Formatting '/var/lib/libvirt/images/f17_sdb.img', fmt=qcow2
size=4294967296 encryption=off cluster_size=65536

- setup xml snippet for new disk (/tmp/sdb.xml)
<disk type='file' device='disk'>
     <driver name='qemu' type='qcow2'/>
     <source file='/var/lib/libvirt/images/f17_sdb.img'/>
     <target dev='sdb' bus='scsi'/>
     <address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>

BTW: any easier method right now? Any planning for virt-manager aware
of virtio_scsi?

- attach the disk to the running domain
# virsh attach-device f17 /tmp/sdb.xml
Device attached successfully

In guest
- rescan
# echo "- - -" > /sys/class/scsi_host/host2/scan

- verify in dmesg
[ 2540.390420] scsi 2:0:0:1: Direct-Access     QEMU     QEMU HARDDISK
  1.0  PQ: 0 ANSI: 5
[ 2540.391195] sd 2:0:0:1: [sdb] 8388608 512-byte logical blocks:
(4.29 GB/4.00 GiB)
[ 2540.391558] sd 2:0:0:1: [sdb] Write Protect is off
[ 2540.391570] sd 2:0:0:1: [sdb] Mode Sense: 63 00 00 08
[ 2540.391787] sd 2:0:0:1: [sdb] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2540.393869] sd 2:0:0:1: Attached scsi generic sg2 type 0
[ 2540.395822]  sdb: unknown partition table
[ 2540.396833] sd 2:0:0:1: [sdb] Attached SCSI disk

- create PV
# pvcreate /dev/sdb
# vgextend vg_f17 /dev/sdb

- vg, lv and fs layout
# vgs
  VG     #PV #LV #SN Attr   VSize  VFree
  vg_f17   2   2   0 wz--n- 12.47g 3.97g

# lvs
  LV      VG     Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert
  lv_root vg_f17 -wi-ao--   8.00g
  lv_swap vg_f17 -wi-ao-- 512.00m

# df -h /
Filesystem                  Size  Used Avail Use% Mounted on
/dev/mapper/vg_f17-lv_root  8.0G  4.6G  3.0G  61% /

- extend lv
# lvextend -l +100%FREE /dev/vg_f17/lv_root
# lvs
  LV      VG     Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert
  lv_root vg_f17 -wi-ao--  11.97g
  lv_swap vg_f17 -wi-ao-- 512.00m

- resize fs
# resize2fs /dev/vg_f17/lv_root
resize2fs 1.42 (29-Nov-2011)
Filesystem at /dev/vg_f17/lv_root is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/vg_f17/lv_root is now 3137536 blocks long.

- layout of fs is now 4Gb more than before
# df -h /
Filesystem                  Size  Used Avail Use% Mounted on
/dev/mapper/vg_f17-lv_root   12G  4.7G  6.8G  41% /

In host while domain running
- resize the disk to 5Gb

# virsh blockresize f17 /var/lib/libvirt/images/f17_sdb.img 5G
Block device '/var/lib/libvirt/images/f17_sdb.img' is resized

In guest
- rescan
# echo 1 > /sys/block/sdb/device/rescan
In dmesg
[ 3248.889379] sd 2:0:0:1: [sdb] 10485760 512-byte logical blocks:
(5.36 GB/5.00 GiB)
[ 3248.889691] sdb: detected capacity change from 4294967296 to 5368709120

- resize PV
# pvresize /dev/sdb
  Physical volume "/dev/sdb" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

- Layout for pv and vg
# pvs
  PV         VG     Fmt  Attr PSize PFree
  /dev/sda2  vg_f17 lvm2 a--  8.50g    0
  /dev/sdb   vg_f17 lvm2 a--  4.97g 1.00g

# vgs
  VG     #PV #LV #SN Attr   VSize  VFree
  vg_f17   2   2   0 wz--n- 13.47g 1.00g

- extend root lv
# lvextend -l +100%FREE /dev/vg_f17/lv_root
  Extending logical volume lv_root to 12.97 GiB
  Logical volume lv_root successfully resized

-resize filesystem
# resize2fs /dev/vg_f17/lv_root
resize2fs 1.42 (29-Nov-2011)
Filesystem at /dev/vg_f17/lv_root is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/vg_f17/lv_root is now 3399680 blocks long.

- show / filesystem now of 13Gb
# df -h /
Filesystem                  Size  Used Avail Use% Mounted on
/dev/mapper/vg_f17-lv_root   13G  4.7G  7.7G  38% /

I successfully added another disk, but placed on another controller
and not as the third disk of the same controller,
Is there any preference to use more disks on the same controller or
creating different controllers, from a qemu/kvm performance point of
view and virtio_scsi?

My steps were

- be sure pci hot plug is enabled in guest (added in /etc/rc.local)
# modprobe shpchp

- add the controller and the disk in one step doesn't work. I got

error: Failed to attach device from /tmp/sdc.xml
error: (device_definition):2: Extra content at the end of the document
<disk type='file' device='disk'>
^

So I create a controller.xml
<controller type='scsi' index='1' model='virtio-scsi'/>

and add it
virsh attach-device f17 /tmp/controller.xml

and then a sdc.xml
<disk type='file' device='disk'>
     <driver name='qemu' type='qcow2'/>
     <source file='/var/lib/libvirt/images/f17_sdc.img'/>
     <target dev='sdc' bus='scsi'/>
     <address type='drive' controller='1' bus='0' target='0' unit='0'/>
</disk>

and add it
virsh attach-device f17 /tmp/sdc.xml

in dmesg of guest when adding the controller
...
[ 4614.355511] scsi3 : Virtio SCSI HBA

In lspci now:
00:06.0 SCSI storage controller: Red Hat, Inc Device 1004
...
00:08.0 SCSI storage controller: Red Hat, Inc Device 1004

- rescan of bus in guest
# echo "- - -" > /sys/class/scsi_host/host3/scan
[ 4751.298514] scsi 3:0:0:0: Direct-Access     QEMU     QEMU HARDDISK
  1.0  PQ: 0 ANSI: 5
[ 4751.299921] sd 3:0:0:0: [sdc] 2097152 512-byte logical blocks:
(1.07 GB/1.00 GiB)
[ 4751.300290] sd 3:0:0:0: [sdc] Write Protect is off
[ 4751.300298] sd 3:0:0:0: [sdc] Mode Sense: 63 00 00 08
[ 4751.300411] sd 3:0:0:0: [sdc] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 4751.301304] sd 3:0:0:0: Attached scsi generic sg3 type 0
[ 4751.303264]  sdc: unknown partition table
[ 4751.303930] sd 3:0:0:0: [sdc] Attached SCSI disk

And I'm able to work

Well done! And thanks for your suggestions

[Paolo Bonzini]

Il 15/06/2012 11:51, Gianluca Cecchi ha scritto:
> BTW: any easier method right now? Any planning for virt-manager aware
> of virtio_scsi?

It should come sooner or later.

In the meanwhile, virt-install supports virtio-scsi quite easily:

virt-install \
   --name=guest1 \
   --controller type=scsi,model=virtio-scsi \
   --disk path=/var/lib/libvirt/images/guest1.img,size=8,bus=scsi,cache=none \
   --vnc \
   --vcpus=2 --ram=2048 \
   --location=http://example1.com/Fedora/17/x86_64/os \
   --os-type=linux \
   --os-variant=rhel6.3

Paolo