Nehalem network performance
Gilboa Davara
gilboad at gmail.com
Sat Mar 6 07:45:42 UTC 2010
Hello Kelvin,
I somehow missed your reply.
On Tue, 2010-02-16 at 15:02 -0500, Kelvin Ku wrote:
> > Did it help?
>
> I switched to an 82576 NIC. The kernel igb driver (version 1.3.16-k2) has
> multiqueue enabled by default with 4 rx and 4 tx queues. I'm running with 4096
> rx ring entries enabled.
>
> The target app is performing well on the igb NIC whereas on the e1000e NIC it
> was missing packets. I say "missing" rather than "dropped" because these
> packets don't show up on any error counters. However, in throughput testing, we
> can't receive faster than 905-910 Mbps whereas we can reliably receive at 950
> Mbps on our older non-Nehalem machines.
You should see the missing package using $ ethtool -s. (Most likely
rx_missed_errors)
> > I usually disable power management.
> > Be advised, that we are using 10GbE cards and not 1GbE, so we are more
> > vulnerable to
> > scaling-the-core-down-right-when-the-cards-starts-flooding-the-hell-out-of-it...
>
> I notice that ASPM is enabled on the 82576 NIC and the PCIe ports. Have you
> disabled ASPM? Disabling C-STATE had no effect on throughput or app
> performance.
At 1Gbps, it shouldn't.
We have ASPM enabled on quad-1GbE and 10GbE without an issue.
>
> I'm going to test with ASPM disabled later today.
>
> >
> > P.S. Please post your complete hardware configuration. (Board, CPU,
> > in-which slot did you put the NIC, etc)
> >
> I've attached dmidecode and lspci output. Here's a summary:
>
> Motherboard: Supermicro X8DTL-iF
> CPU: Single Xeon E5530
> RAM: 3x1GB 1333MHz DDR3 ECC Registered
>
> The 82576 NIC is inserted into a PCIe 2.0 x8 slot.
The weird things is that a two PCI-E 1.0 channels should be more-than to
drive a dual port 1GbE card - let alone 8x/2.0...
I'm stumped.
Side question: Why did you buy an expensive dual socket board, and dual
socket capable CPU (Xeon 55xx) to run a single CPU? If you don't need
the extra codes (or memory), a normal desktop X58/Core i5/i7 will be
just as good and far less expensive.
- Gilboa
More information about the users
mailing list