ixgbe/udev mystery

[Andy Gospodarek]

On Fri, Mar 25, 2011 at 01:46:28PM +0100, Thomas Bendler wrote:
> Hi @all,
> 
> I've setup a F14 box (minimal install) on a Supermicro server. So far, so
> good. After some configuration (mainly automtic stuff done by puppet) and a
> reboot I noticed that only the 1Gb network cards where configured but not
> the 10Gb cards. I didn't even see eth2 and eth3 using ifonfig -a. A look
> into /etc/udev/rules.d/70-persistent-net.rules shows only entries for the
> 1Gb cards:
> 
> # This file was automatically generated by the /lib/udev/write_net_rules
> # program, run by the persistent-net-generator.rules rules file.
> #
> # You can modify it, as long as you keep each rule on a single
> # line, and change only the value of the NAME= key.
> 
> # PCI device 0x8086:0x1096 (e1000e) (custom name provided by external tool)
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
> ATTR{address}=="00:30:48:d7:1c:9e", ATTR{dev_id}=="0x0", ATTR{type}=="1",
> KERNEL=="eth*", NAME="eth0"
> 
> # PCI device 0x8086:0x1096 (e1000e) (custom name provided by external tool)
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
> ATTR{address}=="00:30:48:d7:1c:9f", ATTR{dev_id}=="0x0", ATTR{type}=="1",
> KERNEL=="eth*", NAME="eth1"
> 
> So I've added the 10Gb cards manually (from an Ubuntu installation on an
> equal box):
> 
> # PCI device 0x8086:0x10ec (ixgbe)
>  SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
> ATTR{address}=="00:1b:21:3b:6c:19", ATTR{dev_id}=="0x0", ATTR{type}=="1",
> KERNEL=="eth*", NAME="eth2"
> 
> # PCI device 0x8086:0x10ec (ixgbe)
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
> ATTR{address}=="00:1b:21:3b:6c:18", ATTR{dev_id}=="0x0", ATTR{type}=="1",
> KERNEL=="eth*", NAME="eth3"
> 
> So I also added ifcfg-eth2 and ifcfg-eth3 to /etc/sysconfig/network-scripts
> with something like this (real IP configuration will be done later):
> 
> DEVICE="eth2"
> HWADDR="00:1b:21:3b:6c:19"
> NM_CONTROLLED="yes"
> ONBOOT="no"
> 
> Now the funny stuff starts, after the first reboot the devices eth2 and eth3
> shown up typing ifconfig -a. After the second reboot the eth2 and eth3
> interface disappeared again and I now see in dmesg:
> 
> [    6.159363] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
> version 2.0.62-k2
> [    6.159367] ixgbe: Copyright (c) 1999-2010 Intel Corporation.
> [    6.159425] ixgbe 0000:0c:00.0: enabling device (0000 -> 0002)
> [    6.159435] ixgbe 0000:0c:00.0: PCI->APIC IRQ transform: INT B -> IRQ 17
> [    6.159449] ixgbe 0000:0c:00.0: setting latency timer to 64
> [...]
> [    7.039039] ixgbe 0000:0c:00.0: HW Init failed: -15
> [    7.039077] ixgbe: probe of 0000:0c:00.0 failed with error -15
> [    7.039099] ixgbe 0000:0c:00.1: enabling device (0000 -> 0002)
> [    7.039108] ixgbe 0000:0c:00.1: PCI->APIC IRQ transform: INT A -> IRQ 16
> [    7.039119] ixgbe 0000:0c:00.1: setting latency timer to 64
> [    7.895037] ixgbe 0000:0c:00.1: HW Init failed: -15
> [    7.895056] ixgbe: probe of 0000:0c:00.1 failed with error -15
> 
> Any ideas?
> 

Your devices are not showing up because the driver is failing to load.
Erroe -15 is IXGBE_ERR_RESET_FAILED, so it seems there is either
something wrong with your cards of you are hitting a driver bug.

You should probably turn on dynamic debugging for the ixgbe module and
see if you can figure out what might be going wrong.  A guide for using
dynamic debugging can be found in Documentation/dynamic-debug-howto.txt.