[fedora-virt] Nested KVM help

[Kashyap Chamarthy]

On 08/19/2013 10:13 PM, James Slagle wrote:
> Hi,
> 
> When I try to use nested KVM with Fedora 19, I'm experiencing a hang of the L1
> vm.  I eventually have to do a "Force Off" from virt-manager on L0 since the L1
> vm is completely unresponsive.
> 
> Both my L0 and L1 are F19, x86_64.  L2 doesn't really seem to matter as I can't
> even get the guest to start before L1 completely locks up.  Even just
> attempting to define a L2 guest that uses kvm virtualization locks up L1
> completely.

If you're using a full-blown X, maybe you cay try a minimal server setup across L0, L1,
and L2?

> 
> Here are some more details.
> 
> On L0 (i've rebooted L0 after enabling the nested parameter):
> [root at dublin ~]# uname -a
> Linux dublin 3.10.7-200.fc19.x86_64 #1 SMP Thu Aug 15 23:19:45 UTC 2013 x86_64
> x86_64 x86_64 GNU/Linux
> [root at dublin ~]# cat /sys/module/kvm_intel/parameters/nested 
> Y
> [root at dublin ~]# rpm -q libvirt
> libvirt-1.0.5.5-1.fc19.x86_64
> [root at dublin ~]# lsmod | grep kvm
> kvm_intel             138528  3 
> kvm                   422809  1 kvm_intel
> 
> cpu section of libvirt xml for L1 vm:
>   <cpu mode='custom' match='exact'>
>     <model fallback='allow'>Nehalem</model>
>     <feature policy='require' name='vmx'/>
>   </cpu>

You can also optionally try to do a host CPU pass-through instead of explicitly exposing
the VMX instruction (although that should suffice in most cases).

 <cpu mode='host-passthrough'/>

> 
> 
> On L1:
> [root at localhost jslagle]# uname -a
> Linux localhost.localdomain 3.10.7-200.fc19.x86_64 #1 SMP Thu Aug 15 23:19:45
> UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
> [root at localhost jslagle]# rpm -q libvirt
> libvirt-1.0.5.5-1.fc19.x86_64
> [root at localhost jslagle]# lsmod | grep kvm
> kvm_intel             138528  0 
> kvm                   422809  1 kvm_intel
> 
> 
> I've looked in dmesg output, /var/log/messages, and libvirt logs on both L0 and
> L1 and can't see any errors.
> 
> Any ideas on how to debug this further?

Hm, I've been running some tests on latest Intel Hardware (Haswell), and I have both L1,
and L2 running consistently fine with rawhide kernels.


Here's my set-up details:

  https://github.com/kashyapc/nvmx-haswell/blob/master/SETUP-nVMX.rst

Libvirt XML files for L1 and L2:

  https://github.com/kashyapc/nvmx-haswell/tree/master/tests/libvirt-xmls-for-l1-l2


Hope that helps a bit.


/kashyap