Re: [Fedora-xen] [Xen-devel] Xen PCI passthru supported reset methods (d3d0, FLR, bus reset, link reset)
6:19 a.m.
On 2012-09-17 21:15, Konrad Rzeszutek Wilk wrote:
On Mon, Sep 17, 2012 at 01:42:54PM +0100, Ian Campbell wrote:
> On Mon, 2012-09-17 at 13:32 +0100, Pasi Kärkkäinen wrote:
>> On Mon, Sep 17, 2012 at 01:51:03PM +0200, Robin Axelsson wrote:
>>> There is one thing I wonder though when it comes to PCI passthrough:
>>>
>>> Can Xen reset hardware through the d3d0 in the ACPI interface and/or
>>> through a 'bus reset' or a 'link reset'? Or can it reset
hardware
>>> that is marked for passthrough only through FLR?
>>>
>>> For details see e.g.
>>> http://www.vmware.com/files/pdf/techpaper/vsp_4_vmdirectpath_host.pdf
>>>
>> I added xen-devel to the CC-list.
>> Hopefully someone there can reply this question.
> With a pvops dom0 Xen resets devices by writing to its "reset" node in
> sysfs so it will reset the device using whatever method the dom0 kernel
> supports for that device.
And if you use Xen PCI-back it has this enabled so you don't even
need the 'reset' functionality.
> The version of Linux I have to hand has, in __pci_dev_reset, calls to
> the following in this order and stops after the first one which
> succeeds:
> * pci_dev_specific_reset (AKA per device quirks)
> * pcie_flr
> * pci_af_flr
> * pci_pm_reset
> * pci_parent_bus_reset
>
> See drivers/pci/pci.c in the kernel for more info.
>
> IIRC classic Xen kernels had similar code in pciback, although I don't
> know which specific sets of actions or in which order they were tried.
>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel(a)lists.xen.org
> http://lists.xen.org/xen-devel
>
.
That sounds like great news, that means that FLR is not a requirement to
successfully pass through hardware without errors, as is stated in the
VTdHowTo page. So it seems that the VTdHowTo page needs to be updated
with this information.
6:39 a.m.
New subject: [Xen-devel] Xen PCI passthru supported reset methods (d3d0, FLR, bus reset, link reset)
On Tue, Sep 18, 2012 at 01:19:31PM +0200, Robin Axelsson wrote:
>>With a pvops dom0 Xen resets devices by writing to its
"reset" node in
>>sysfs so it will reset the device using whatever method the dom0 kernel
>>supports for that device.
>And if you use Xen PCI-back it has this enabled so you don't even
>need the 'reset' functionality.
>>The version of Linux I have to hand has, in __pci_dev_reset, calls to
>>the following in this order and stops after the first one which
>>succeeds:
>> * pci_dev_specific_reset (AKA per device quirks)
>> * pcie_flr
>> * pci_af_flr
>> * pci_pm_reset
>> * pci_parent_bus_reset
>>
>>See drivers/pci/pci.c in the kernel for more info.
>>
>>IIRC classic Xen kernels had similar code in pciback, although I don't
>>know which specific sets of actions or in which order they were tried.
>>
>>Ian.
>>
>
That sounds like great news, that means that FLR is not a
requirement to successfully pass through hardware without errors, as
is stated in the VTdHowTo page. So it seems that the VTdHowTo page
needs to be updated with this information.
Do you want to update the wiki page? :)
-- Pasi
10:02 a.m.
New subject: [Xen-devel] Xen PCI passthru supported reset methods (d3d0, FLR, bus reset, link reset)
On 2012-09-18 13:39, Pasi Kärkkäinen wrote:
On Tue, Sep 18, 2012 at 01:19:31PM +0200, Robin Axelsson wrote:
>>> With a pvops dom0 Xen resets devices by writing to its "reset" node
in
>>> sysfs so it will reset the device using whatever method the dom0 kernel
>>> supports for that device.
>> And if you use Xen PCI-back it has this enabled so you don't even
>> need the 'reset' functionality.
>>> The version of Linux I have to hand has, in __pci_dev_reset, calls to
>>> the following in this order and stops after the first one which
>>> succeeds:
>>> * pci_dev_specific_reset (AKA per device quirks)
>>> * pcie_flr
>>> * pci_af_flr
>>> * pci_pm_reset
>>> * pci_parent_bus_reset
>>>
>>> See drivers/pci/pci.c in the kernel for more info.
>>>
>>> IIRC classic Xen kernels had similar code in pciback, although I don't
>>> know which specific sets of actions or in which order they were tried.
>>>
>>> Ian.
>>>
> That sounds like great news, that means that FLR is not a
> requirement to successfully pass through hardware without errors, as
> is stated in the VTdHowTo page. So it seems that the VTdHowTo page
> needs to be updated with this information.
>
Do you want to update the wiki page? :)
-- Pasi
.
I wouldn't mind updating the Wiki but I don't have the authority and
perhaps not the knowledge.
Before announcing "official" support for non-FLR hardware I think some
testing or testimonials would be in place. When I was experimenting with
IOMMU on a server a little over half a year ago right before sending it
out for regular use, the xen pciback drivers were not properly rolled
out on the paravirt ops kernel. But this probably has improved by then
and I do happen to have IOMMU capable hardware available for testing
once again.
* When reading into the pci.c code I think I understand the "pcie_flr"
and "pci_pm_reset" methods/functions but I don't 100% understand the
other functions used above to reset hardware. I guess I could figure
that "pci_parent_bus_reset" trigs a bus reset, but I can't tell the
difference between "pci_dev_specific_reset" and "pcie_flr" or what
'af'
means in "pci_af_flr".
* Another thing that I've always been wondering is whether d3d0 trigs a
reset in hardware that takes power from an auxiliary input and don't
rely on the power provided by the PCI bus (such as GPUs and some
soundcards such as the Asus Xonar Essence STX). I read about a guy who
managed to pass through a non-FLR radeon card to a windows guest by
using the "Safe remove" feature in the Windows domU which perhaps
trigged a d3d0 reset through the ACPI framework somehow. Here's what he
wrote:
"I've never had Xen throw errors at me regarding FLR for the 5850, but
some recent experience suggests that FLR doesn't quite behave correctly
when it's initiated by pciback... really screwy stuff. There have been a
lot of threads between me and a few other people on the Xen-Users
mailing list regarding the issue of PCI passthrough for Radeon 5XXX and
6XXX series cards to Windows HVM guests.
Radeon cards seem to work properly when they're being used after they're
first used by a DomU following a restart of Dom0, then suffer
performance problems if DomU is rebooted and Dom0 is not. The solution
is to use the Windows "Safe Remove" feature on the video card. That
forces a proper FLR (for some reason) and makes it work right.
Aside from that, I can't get the 5850 to coexist with the GPLPV
package.... and I wish I knew why. It just BSODs every time. :("
He wrote this about 4 months ago so some things may have changed since then.
So unless more information is divulged in this matter, at least the Wiki
could be updated with an honorable mention of these other reset methods
instead of just saying that non-FLR hardware is a no-go for PCI passthrough.
Robin.
4237
days inactive
4237
days old
2 comments
2 participants
participants (2)
-
Pasi Kärkkäinen -
Robin Axelsson