Multiple external zone interfaces

[Larry M. Smith]

I am seeing an issue with my configuration that I can't seem to wrap my 
head around.  With multiple interfaces in the the external zone, using 
masquerade, only one is usable.

--8<-------------------------------------------------------------
[root@gate01 ~]# firewall-cmd --get-active-zones
external
   interfaces: eno2 eno1
internal
   interfaces: eno4 eno3

[root@gate01 ~]# firewall-cmd --zone=external --list-all
external (active)
   target: default
   icmp-block-inversion: no
   interfaces: eno1 eno2
   sources:
   services: ssh
   ports:
   protocols:
   forward: no
   masquerade: yes
   forward-ports:
   source-ports:
   icmp-blocks:
   rich rules:

[root@gate01 ~]# ip addr show dev eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP 
group default qlen 1000
     link/ether de:ad:be:ef:ca:fe brd ff:ff:ff:ff:ff:ff permaddr 
3c:ec:ef:54:bc:78
     inet 192.0.2.203/25 brd 192.0.2.255 scope global dynamic 
noprefixroute eno1

[root@gate01 ~]# ip addr show dev eno2
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP 
group default qlen 1000
     link/ether ca:fe:be:ef:ca:fe brd ff:ff:ff:ff:ff:ff
     inet 198.51.100.179/26 brd 198.51.100.191 scope global dynamic 
noprefixroute eno2

[root@gate01 ~]# ip route
default via 192.0.2.129 dev eno1 proto dhcp metric 104
default via 198.51.100.129 dev eno2 proto dhcp metric 105
198.51.100.128/26 dev eno2 proto kernel scope link src 198.51.100.179 
metric 105
192.0.2.128/25 dev eno1 proto kernel scope link src 192.0.2.203 metric 104
192.168.230.0/23 dev eno3 proto kernel scope link src 192.168.230.3 
metric 102
192.168.232.0/23 dev eno4 proto kernel scope link src 192.168.232.3 
metric 100
--8<-------------------------------------------------------------

With this configuration, only the interface with the lowest metric 
default route is usable;

--8<-------------------------------------------------------------
[chains@imp ~]$ ping -c 2 192.0.2.203
PING 192.0.2.203 (192.0.2.203) 56(84) bytes of data.
64 bytes from 192.0.2.203: icmp_seq=1 ttl=42 time=56.8 ms
64 bytes from 192.0.2.203: icmp_seq=2 ttl=42 time=122 ms

--- 192.0.2.203 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 56.813/89.766/122.719/32.953 ms

[chains@imp ~]$ ping -c 2 198.51.100.179
PING 198.51.100.179 (198.51.100.179) 56(84) bytes of data.

--- 198.51.100.179 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
--8<-------------------------------------------------------------

However, if I make if eno2 the lower metric interface then eno2 is 
ping-able and eno1 isn't.

.. Anyone have any idea what I might be doing wrong?

SgtChains

(Distro is "Rocky Linux release 8.5" and Firewalld is "0.9.3")

[Larry M. Smith]

On 4/17/2022, Larry M. Smith wrote:
(snip)

Never mind... I appear to figure it out.  It was an internal routing 
issue and nothing to do with firewalld.

cat<<EOF>>/etc/iproute2/rt_tables
1 eno1
2 eno2
EOF

--8<-------------------------------------------------------------
[root@gate01 ~]# ip rule add from 192.0.2.203 table eno1
[root@gate01 ~]# ip route add default via 192.0.2.129 dev eno1 table eno1

[root@gate01 ~]# ip rule add from 198.51.100.179 table eno2
[root@gate01 ~]# ip route add default via 198.51.100.129 dev eno2 table eno2
--8<-------------------------------------------------------------

Now I can ping both interfaces from a remote host.  Sorry for the trouble.

SgtChains