Hi everyone,
(Originally sent to the Fedora Kernel list but since I wasn't subscribed it was blocked.)
I run a Fedora IoT based router for my network which performs IPv4 NAT for my network out to the internet.
After a recent upgrade to F44 HTTP/3/QUIC based connections started to fail reliably.
With the assistance of tcpdump I was able to trace it to what I believe is the IPv4 length headers getting corrupted by the router specifically when forwarding packets. It does not appear to happen if the connection originates from the router.
If generic-receive-offload is enabled for the router WAN interface I am able to reproduce the failures. If I disable generic-receive-offload I am able to utilize HTTP/3 connections without any failures.
The ethernet card is a built in Intel I226-V (rev 4).
ethtool -i reports the following:
driver: igc version: 7.0.6-200.fc44.x86_64 firmware-version: 2022:889d expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes
So far I have been able to narrow it down to working under 6.19.10-300.fc44 and failing under 7.0.4-200.fc44.x86_64.
Finally the command I am using to test with is the following:
curl \ -v \ --resolve media.cnn.com:443:151.101.67.5 \ --http3-only \ -H 'accept: */*' \ -H 'accept-language: en-US,en;q=0.9' \ -H 'cache-control: no-cache' \ -H 'range: bytes=0-' \ --output /dev/null \ 'https://media.cnn.com/api/v1/loops/stellar/prod/gettyimages-487272667.mp4?c=...'
Due to it being IoT I'm a little unsure of the best way to go about trying to bisect this so any input or recommendations would be appreciated unless there is already a known issue.
Thanks, Kevin
Hi Kevin,
(Originally sent to the Fedora Kernel list but since I wasn't subscribed it was blocked.)
I run a Fedora IoT based router for my network which performs IPv4 NAT for my network out to the internet.
After a recent upgrade to F44 HTTP/3/QUIC based connections started to fail reliably.
With the assistance of tcpdump I was able to trace it to what I believe is the IPv4 length headers getting corrupted by the router specifically when forwarding packets. It does not appear to happen if the connection originates from the router.
If generic-receive-offload is enabled for the router WAN interface I am able to reproduce the failures. If I disable generic-receive-offload I am able to utilize HTTP/3 connections without any failures.
The ethernet card is a built in Intel I226-V (rev 4).
I seem the remember those NICs generally have issues with GRO in general, although it's been a while since I've dealt with them. I wonder if there's been a default that has changed between the 6.19 -> 7.0 kernel around GRO, or a default between F43 and F44.
When running a 6.19 kernel can you check if GRO is enabled by default on that NIC/kernel? Does it show the same issues if GRO is enabled on the 6.19 kernel?
ethtool -i reports the following:
driver: igc version: 7.0.6-200.fc44.x86_64 firmware-version: 2022:889d expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes
So far I have been able to narrow it down to working under 6.19.10-300.fc44 and failing under 7.0.4-200.fc44.x86_64.
Finally the command I am using to test with is the following:
curl \ -v \ --resolve media.cnn.com:443:151.101.67.5 \ --http3-only \ -H 'accept: */*' \ -H 'accept-language: en-US,en;q=0.9' \ -H 'cache-control: no-cache' \ -H 'range: bytes=0-' \ --output /dev/null \ 'https://media.cnn.com/api/v1/loops/stellar/prod/gettyimages-487272667.mp4?c=...'
Due to it being IoT I'm a little unsure of the best way to go about trying to bisect this so any input or recommendations would be appreciated unless there is already a known issue.
Thanks, Kevin -- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://forge.fedoraproject.org/infra/tickets/issues/new
On Sun, May 17, 2026 at 05:23:38PM +0100, Peter Robinson wrote:
Hi Peter,
If generic-receive-offload is enabled for the router WAN interface I am able to reproduce the failures. If I disable generic-receive-offload I am able to utilize HTTP/3 connections without any failures.
The ethernet card is a built in Intel I226-V (rev 4).
I seem the remember those NICs generally have issues with GRO in general, although it's been a while since I've dealt with them. I wonder if there's been a default that has changed between the 6.19 -> 7.0 kernel around GRO, or a default between F43 and F44.
When running a 6.19 kernel can you check if GRO is enabled by default on that NIC/kernel? Does it show the same issues if GRO is enabled on the 6.19 kernel?
I rebased to 43.20260424.0 with the 6.19.13-200.fc43.x86_64 kernel and checked that generic-receive-offload defaults to on in that version and it does. I also verified that I do not see the same issues with that version.
Thanks, Kevin
Hi Kevin,
If generic-receive-offload is enabled for the router WAN interface I am able to reproduce the failures. If I disable generic-receive-offload I am able to utilize HTTP/3 connections without any failures.
The ethernet card is a built in Intel I226-V (rev 4).
I seem the remember those NICs generally have issues with GRO in general, although it's been a while since I've dealt with them. I wonder if there's been a default that has changed between the 6.19 -> 7.0 kernel around GRO, or a default between F43 and F44.
When running a 6.19 kernel can you check if GRO is enabled by default on that NIC/kernel? Does it show the same issues if GRO is enabled on the 6.19 kernel?
I rebased to 43.20260424.0 with the 6.19.13-200.fc43.x86_64 kernel and checked that generic-receive-offload defaults to on in that version and it does. I also verified that I do not see the same issues with that version.
OK. A look at the changes in 7.0 [1] for that drive for 7.0 are fairly minimal and mostly are either tree wide, or fixes for XDP features which you've not mentioned your router is using. I wonder where else it may lie.
[1] git log v6.19..v7.0 --no-merges drivers/net/ethernet/intel/igc/
Hi Peter,
OK. A look at the changes in 7.0 [1] for that drive for 7.0 are fairly minimal and mostly are either tree wide, or fixes for XDP features which you've not mentioned your router is using. I wonder where else it may lie.
[1] git log v6.19..v7.0 --no-merges drivers/net/ethernet/intel/igc/
I haven't been able to get a kernel rebuilt successfully under IoT but I did notice a new kernel release in the IoT compose.
I rebased to that version and it appears to have resolved the problem that I was seeing!
For reference that is 7.0.8-200.fc44.x86_64 in 44.20260518.0 and it does not show the same problem that I saw in earlier 7.x kernels.
Thank you for looking in to it!
- Kevin
-
Kevin Anderson -
Peter Robinson