vfio Fedora 19

[James Hubbard]

Is vfio working under Fedora 19?  I'm up to date and have the virt-preview
repo installed.  I had to use modprobe vfio-pci so that /dev/vfio/vifo
would show up.

This is the error that I'm getting.
error: internal error: Invalid device 0000:83:00.0 driver file
 /sys/bus/pci/devices/0000:83:00.0/driver is not a symlink

I would have expected iommu_group rather than driver at the end of that.

BTW, this is not generating an error message in virt-manager like it was
prior to updating to virt-preview.

[Cole Robinson]

On 10/15/2013 10:47 PM, James Hubbard wrote:
> Is vfio working under Fedora 19?  I'm up to date and have the virt-preview
> repo installed.  I had to use modprobe vfio-pci so that /dev/vfio/vifo would
> show up.
>
> This is the error that I'm getting. 
> error: internal error: Invalid device 0000:83:00.0 driver file
>  /sys/bus/pci/devices/0000:83:00.0/driver is not a symlink
>
> I would have expected iommu_group rather than driver at the end of that.

Sounds like your system doesn't have an iommu or it isn't enabled, confirm
with the 'verify' steps here:

http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM#Assigni...
> BTW, this is not generating an error message in virt-manager like it was prior
> to updating to virt-preview.

Make sure you restart virt-manager after updating. If after that you can see
the error in the --debug output but not from the UI, please file a bug and
make sure to mention your virt-manager version.

- Cole

[James Hubbard]

On Wed, Oct 16, 2013 at 7:31 AM, Cole Robinson  wrote:
> Sounds like your system doesn't have an iommu or it isn't enabled, confirm
> with the 'verify' steps here:
>
> http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM#Assigni...
> > BTW, this is not generating an error message in virt-manager like it was
>
> prior
> > to updating to virt-preview.
>
> Make sure you restart virt-manager after updating. If after that you can
> see
> the error in the --debug output but not from the UI, please file a bug and
> make sure to mention your virt-manager version.

The setting intel_iommu is enabled.  I was able to get the Intel i350 cards
setup with SR-IOV.  The DMAR and IOMMU items show up fine.

[Cole Robinson]

On 10/16/2013 09:03 AM, James Hubbard wrote:
> On Wed, Oct 16, 2013 at 7:31 AM, Cole Robinson <[e-mail]
> > wrote:
> Sounds like your system doesn't have an iommu or it isn't enabled, confirm
> with the 'verify' steps here:
>
> http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM#Assigning_device_to_guest
>
> > BTW, this is not generating an error message in virt-manager like it was
> prior
> > to updating to virt-preview.
> >
>
> Make sure you restart virt-manager after updating. If after that you can see
> the error in the --debug output but not from the UI, please file a bug and
> make sure to mention your virt-manager version.
>
>
> The setting intel_iommu is enabled.  I was able to get the Intel i350 cards
> setup with SR-IOV.  The DMAR and IOMMU items show up fine.

Are there any dmesg messages about vfio? The module was autoloaded for me on
f19, that lack of autoload coupled with the libvirt error make me think
something is weird with your hardware, or at least the kernel thinks so.

Regardless, please file a libvirt bug so we can track this.

Thanks,
Cole

[James Hubbard]

On Wed, Oct 16, 2013 at 9:11 AM, Cole Robinson  wrote:
> On 10/16/2013 09:03 AM, James Hubbard wrote:
> > On Wed, Oct 16, 2013 at 7:31 AM, Cole Robinson <[e-mail]
> > > wrote:
> > Sounds like your system doesn't have an iommu or it isn't enabled,
>
> confirm
> > with the 'verify' steps here:
>
> http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM#Assigni...
> > > BTW, this is not generating an error message in virt-manager like
>
> it was
> > prior
> > > to updating to virt-preview.
> > >
> >
> > Make sure you restart virt-manager after updating. If after that you
>
> can see
> > the error in the --debug output but not from the UI, please file a
>
> bug and
> > make sure to mention your virt-manager version.
> >
> >
> > The setting intel_iommu is enabled.  I was able to get the Intel i350
>
> cards
> > setup with SR-IOV.  The DMAR and IOMMU items show up fine.
>
> Are there any dmesg messages about vfio? The module was autoloaded for me
> on
> f19, that lack of autoload coupled with the libvirt error make me think
> something is weird with your hardware, or at least the kernel thinks so.
>
> Regardless, please file a libvirt bug so we can track this.

I'm not seeing any error messages in dmesg when attempting to start.  When
I do modprobe vfio-pci I do see:
VFIO - User Level meta-driver version: 0.3

It wouldn't suprise me if it was a hardware problem.  The motherboard is a
SuperMicro X9DRD-iF with a version 1.0 firmware.  I've seen some flakiness
with another SuperMicro motherboard when intel_iommu=on is set.

Thanks for the help
James Hubbard

[James Hubbard]

On Wed, Oct 16, 2013 at 10:04 AM, James Hubbard wrote:
> On Wed, Oct 16, 2013 at 9:11 AM, Cole Robinson wrote:
> > On 10/16/2013 09:03 AM, James Hubbard wrote:
> > > On Wed, Oct 16, 2013 at 7:31 AM, Cole Robinson <[e-mail]
> > > > wrote:
> > > Sounds like your system doesn't have an iommu or it isn't enabled,
> >
> > confirm
> > > with the 'verify' steps here:
> >
> > http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM#Assigni...
> > > > BTW, this is not generating an error message in virt-manager like
> >
> > it was
> > > prior
> > > > to updating to virt-preview.
> > > >
> > >
> > > Make sure you restart virt-manager after updating. If after that
> >
> > you can see
> > > the error in the --debug output but not from the UI, please file a
> >
> > bug and
> > > make sure to mention your virt-manager version.
> > >
> > >
> > > The setting intel_iommu is enabled.  I was able to get the Intel i350
> >
> > cards
> > > setup with SR-IOV.  The DMAR and IOMMU items show up fine.
> >
> > Are there any dmesg messages about vfio? The module was autoloaded for me
> > on
> > f19, that lack of autoload coupled with the libvirt error make me think
> > something is weird with your hardware, or at least the kernel thinks so.
> >
> > Regardless, please file a libvirt bug so we can track this.
>
> I'm not seeing any error messages in dmesg when attempting to start.  When
> I do modprobe vfio-pci I do see:
> VFIO - User Level meta-driver version: 0.3
>
> It wouldn't suprise me if it was a hardware problem.  The motherboard is a
> SuperMicro X9DRD-iF with a version 1.0 firmware.  I've seen some flakiness
> with another SuperMicro motherboard when intel_iommu=on is set.

I upgraded the firmware, no change.  I added two bug reports.
https://bugzilla.redhat.com/show_bug.cgi?id=1020326
https://bugzilla.redhat.com/show_bug.cgi?id=1020336