I'm running Fedora 30 on a somewhat older PC:
$ uname -a Linux pc.localdomain 5.4.14-100.fc30.x86_64 #1 SMP Thu Jan 23 13:19:57 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
I am experiencing just one minor issue having to do with throughput on the PC's built-in Ethernet interface:
# lspci -v -s 04:00.0 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) Subsystem: Gigabyte Technology Co., Ltd Onboard Ethernet Flags: bus master, fast devsel, latency 0, IRQ 18, NUMA node 0 I/O ports at ce00 [size=256] Memory at fd9ff000 (64-bit, prefetchable) [size=4K] Memory at fd9f8000 (64-bit, prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable+ Count=4 Masked- Capabilities: [d0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Virtual Channel Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00 Kernel driver in use: r8169 Kernel modules: r8169
Until recently, the iperf3 network benchmark reported 940 Mbits/sec throughput in both directions but now for some unknown reason outgoing traffic is being throttled to 870 Mbits/sec:
$ iperf3 -c backup Connecting to host backup, port 5201 [ 5] local 192.168.4.6 port 43050 connected to 192.168.4.15 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 105 MBytes 884 Mbits/sec 0 300 KBytes [ 5] 1.00-2.00 sec 105 MBytes 877 Mbits/sec 0 328 KBytes [ 5] 2.00-3.00 sec 104 MBytes 874 Mbits/sec 0 328 KBytes [ 5] 3.00-4.00 sec 104 MBytes 874 Mbits/sec 0 345 KBytes [ 5] 4.00-5.00 sec 104 MBytes 873 Mbits/sec 0 345 KBytes [ 5] 5.00-6.00 sec 105 MBytes 879 Mbits/sec 0 345 KBytes [ 5] 6.00-7.00 sec 104 MBytes 873 Mbits/sec 0 345 KBytes [ 5] 7.00-8.00 sec 104 MBytes 873 Mbits/sec 0 345 KBytes [ 5] 8.00-9.00 sec 104 MBytes 873 Mbits/sec 0 345 KBytes [ 5] 9.00-10.00 sec 104 MBytes 873 Mbits/sec 0 345 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.02 GBytes 875 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 1.02 GBytes 874 Mbits/sec receiver
but incoming bandwidth is 940 Mbits/sec:
$ iperf3 -c backup -R Connecting to host backup, port 5201 Reverse mode, remote host backup is sending [ 5] local 192.168.4.6 port 43056 connected to 192.168.4.15 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 112 MBytes 941 Mbits/sec [ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec [ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec [ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec [ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 180 sender [ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver
iperf Done.
The incoming throughput confirms that the Ethernet port is capable of 940 Mbits/sec.
My PC and 'backup' are connected to the same managed 1GigE switch (Ubiquiti EdgeSwitch 8). To rule out a switch configuration issue, I ran the same iperf3 test while the devices were connected to an unmanaged switch and got the same results.
The host named 'backup' tests at 940 Mbits/sec in both directions when tested against another Fedora 30 Linux PC of similar vintage so I don't think 'backup' (a Linux-based Synology NAS) is the issue.
'ethtool' shows that my PC's Ethernet auto-negotiates 1Gig mode:
# ethtool enp3s0 Settings for enp3s0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Link partner advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Link partner advertised pause frame use: No Link partner advertised auto-negotiation: Yes Link partner advertised FEC modes: Not reported Speed: 1000Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000033 (51) drv probe ifdown ifup Link detected: yes
The Linux network interface matches up with the other Linux PC mentioned above. Note the MTU is 1500 (as it is on the 'backup' NAS).
# ip a show dev enp3s0 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 94:de:80:21:61:12 brd ff:ff:ff:ff:ff:ff inet 192.168.4.6/24 brd 192.168.4.255 scope global dynamic noprefixroute enp3s0 valid_lft 86075sec preferred_lft 86075sec inet6 fe80::7c0a:4567:cd0f:13db/64 scope link noprefixroute valid_lft forever preferred_lft forever
# ip r default via 192.168.4.1 dev enp3s0 proto dhcp metric 100 192.168.4.0/24 dev enp3s0 proto kernel scope link src 192.168.4.6 metric 100
I did recently configure a smart queue policy on my LAN's Ubiquiti EdgeRouter 4's WAN interface but I doubt this contributing to the issue for three reasons: (1) the smart queue policy is bound to the router's WAN interface which is a different interface than the one that trunks to my PC's switch; (2) my benchmark test is between two devices on the same switch plus the switch's dashboard GUI makes it obvious that the traffic during the test doesn't go out via the trunk uplink to the router and (3) removing the router's smart queue policy then running the PC -> NAS iperf3 benchmark produces the same throughput numbers. Also, I recently configured the PC to go into suspend after 90 minutes and to set up a magic packet wake-on-lan before it suspends. I've tried backing off these changes followed by a cold reboot...no change in the iperf3 results.
If this is a Linux kernel issue, it would have to have been introduced several kernel releases ago because if I boot into the PC's oldest kernel, 5.4.10-100.fc30.x86_64, I still suffer from the slower outgoing throughput. So, either this issue began with a Linux kernel < 5.4.10 or it's due to a configuration change I made but lost track of.
Have I run into a known, or unreported, issue with the newer Linux kernels or the r8169 module in particular? If not, what might I look into next to try to pinpoint what's causing the reduction in outgoing throughput? Any other suggestions?
Thanks, Dave