Am 17.06.2024 um 12:53:31 Uhr schrieb R C:
Well, I am talking about firewalld, RHEL8. Regardless of SPI. My reasoning is, a firewall needs to check pretty much every packet a node receives.
If you do SPI (default in many cases and necessary in many situations), also outgoing packages have to be investigated to be tracked to match the incoming (allow related) packages.
A node, compute or data node, receives a lot of traffic (That's why Infiniband is used, IB) and a compute node uses a lot of cycles, typicaly one tries to come as close to the number of flops to be used (also threads/ranks and core use). So every incoming packet needs to be done something firewall wise (which takes cycles) so that has an impact on bandwith between nodes.
Is has impact on the bandwidth if too less CPU cycles are available.
On the other hand machines "running at/close to max" don't have of cycles to spare. So a busy compute node with a lot of traffic mightbe seriously impacted(?). Are there any available metrics, or what would be a good way to 'predict' that?
Test maximum throughput to many different destinations and measure CPU cycles.