Policing traffic between and/or originated by secondary IP addresses

[Mai Ling]

Hi,

I am trying to filter traffic between pasta rootless podman
containers, using manual setup without any managed solution, on an
EL9.2 clone.
The simple case I wish to complete is to permit traffic to api only to
it's 8088 port and only from web source, and do not allow web to
connect to anywhere else, and neither api.

For this, I have setup two containers, named web and api, and given
them the addresses
10.1.1.211 web
10.1.1.221 api

I've setup their networking using the pasta software,using podman
create -h api --network=pasta:-4,-a,10.1.1.221,--map-gw,--no-udp,--no-dhcp,--no-ndp,--no-ra,--no-dhcpv6,-I,eth0,-t,10.1.1.221/1882:80
(and similarly web 10.1.1.211/1802:80)

I am running each of them into their own host OS username and I've
choosen the same number for their UID as well as for the TCP listening
port number of their own containers.

$ id -u web
1802
$ id -u api
1882

In firewalld I have configured masquerading each UID into their own
outgoing IP address:

$ sudo firewall-cmd --direct --get-all-rules
ipv4 nat POSTROUTING 0 -m owner --uid-owner 1802 -j SNAT --to-source 10.1.1.211
ipv4 nat POSTROUTING 0 -m owner --uid-owner 1882 -j SNAT --to-source 10.1.1.221

The IP addresses are already assigned on the network interface
$ nmcli c s public | grep ipv4.address
ipv4.addresses:                         10.1.1.223/24, 10.1.1.211/24,
10.1.1.221/24

I have created a zone for each source IP address, and two policies:
one web to api, one web to anywhere:

$ sudo firewall-cmd --get-active-zones
fromapi
  sources: 10.1.1.221
fromweb
  sources: 10.1.1.211

$ sudo firewall-cmd --zone=fromapi --list-all
fromapi (active)
  target: default
  icmp-block-inversion: no
  interfaces:
  sources: 10.1.1.221
  services:
  ports:
  protocols:
  forward: no
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

$ sudo firewall-cmd --zone=fromweb --list-all
fromweb (active)
  target: default
  icmp-block-inversion: no
  interfaces:
  sources: 10.1.1.211
  services:
  ports:
  protocols:
  forward: no
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

$ sudo firewall-cmd --get-policies
allow-host-ipv6 web2any web2api

$ sudo firewall-cmd --info-policy=web2any
web2any (active)
  priority: -1
  target: REJECT
  ingress-zones: fromweb
  egress-zones: ANY
  services:
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

Note above changing the default policy to REJECT

$ sudo firewall-cmd --info-policy=web2api
web2api (active)
  priority: -1
  target: CONTINUE
  ingress-zones: fromweb
  egress-zones: fromapi
  services:
  ports: 8088/tcp
  protocols:
  masquerade: no
  forward-ports:
        port=8088:proto=tcp:toport=1882:toaddr=10.1.1.221
  source-ports:
  icmp-blocks:
  rich rules:

Note above I am doing the port forward because I intend to setup a
second host and I want to standardize across hosts what port would the
API run, while allowing to keep using the uid=port pair numbers using
podman

The resulting nft list ruleset is quite big, pasted here:
https://paste.sh/3_ejeq92#bt4HpZy-Gg0IIfN6wxleFU5W

Now, to testing.

First issue:
I enter web shell and run:
[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 api:1882
api 10.1.1.221 port local 1882 port mapped 8088 # api answer produced
by a echo + netcat script

On tcpdump however, I see the response from api going to HOST's
primary IP address (10.1.1.223) instead of returning to web IP
address. Not sure what is the cause and how this affects the goal, but
mentioning anyway:

$ sudo tcpdump -qnni any not port 22
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot
length 262144 bytes
20:01:46.369277 lo    In  IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.369321 lo    In  IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.369345 lo    In  IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.369478 lo    In  IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 72
20:01:46.369496 lo    In  IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.404545 lo    In  IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 97
20:01:46.404584 lo    In  IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.404798 lo    In  IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0
20:01:46.405043 lo    In  IP 10.1.1.211.50922 > 10.1.1.221.1882: tcp 0
20:01:46.405066 lo    In  IP 10.1.1.221.1882 > 10.1.1.223.50922: tcp 0

Attempting however to connect to the mapped port - fails, and I see no
traffic in tcpdump:
[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 api:8088
curl: (28) Connection timed out after 1001 milliseconds

Second, attempting to connect from web to anywhere, such as to a
nearby proxy, unexpectedly succeeds:

[root@web /]# http_proxy='' ALL_PROXY='' curl -m 1 10.1.1.252:3128

<html><head>
[...]

tcpdump:
20:07:22.747293 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 0
20:07:22.747466 ens192 B   ARP, Request who-has 10.1.1.211 tell
10.1.1.252, length 46
20:07:22.747478 ens192 Out ARP, Reply 10.1.1.211 is-at
00:0c:36:e2:8a:7d, length 28
20:07:22.747560 ens192 In  IP 10.1.1.252.3128 > 10.1.1.211.53902: tcp 0
20:07:22.747595 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 0
20:07:22.747872 ens192 Out IP 10.1.1.211.53902 > 10.1.1.252.3128: tcp 83
[...]

Questions:
1) Why does the response from api goes to destination being primary IP
of the host, instead of the IP alias of the web container?
2) Why does web can initiate outgoing to anywhere?

I tried raising the priority for web2api rule to -10 but despite
"success" status, it seems to have not changed:

$ sudo firewall-cmd --permanent --policy=web2api --set-priority=-10
success
$ sudo firewall-cmd  --policy=web2api --set-priority=-10
success
$ sudo firewall-cmd --info-policy=web2api
web2api (active)
  priority: -1

not even after --reload (maybe is this an issue fixed in a later release?)

changed it by editing the xml:
$ sudo firewall-cmd --info-policy=web2api
web2api (active)
  priority: -10

Yet still web can make outgoing requests other than what expected.

I am fairly new at understanding zones and policies, and despite
having used firewalld before and reading and re-reading the docs,
things are still a bit fuzzy for me. For example it took many
re-reading to conclude that egress and ingress mean the direction of
traffic in relation to the firewall itself and not in relation to the
interface (which is exactly in the opposite direction), or that I can
have a zone without an interface and only using a source IP; but can I
use that zone to filter traffic destined _to_ that IP when used in a
policy as an egress zone, or should I explicitly configure that in a
rich rule?

[Mai Ling]

> $ sudo firewall-cmd --get-active-zones
> fromapi
>   sources: 10.1.1.221
> fromweb
>   sources: 10.1.1.211

an incomplete mouse-pasted here missed the interfaces' zones

internal
  interfaces: ens224
public
  interfaces: ens192

[Andrei Borzenkov]

On 15.10.2023 20:31, Mai Ling wrote:
> Why does the response from api goes to destination being primary IP
>
> of the host, instead of the IP alias of the web container?

Reply to TCP connection always goes to the source IP address. If 
application did not explicitly select specific source IP when 
establishing connection, kernel picks up one of suitable addresses. I 
suppose kernel simply takes the first address assigned to the interface.

I have no idea how traffic from containers is mapped to the available 
addresses, but I am sure firewalld is not involved in this decision.
> Why does web can initiate outgoing to anywhere?

Educated guess - because source address does not match your zone definition.

[Mai Ling]

On Sun, Oct 15, 2023 at 9:04 PM Andrei Borzenkov  wrote:
> On 15.10.2023 20:31, Mai Ling wrote:
> > Why does the response from api goes to destination being primary IP
> >
> > of the host, instead of the IP alias of the web container?
>
> Reply to TCP connection always goes to the source IP address. If
> application did not explicitly select specific source IP when
> establishing connection, kernel picks up one of suitable addresses. I
> suppose kernel simply takes the first address assigned to the interface.
>
> I have no idea how traffic from containers is mapped to the available
> addresses, but I am sure firewalld is not involved in this decision.

in podman rootless, pasta is a process spawned when starting the
container. it forwards packets from container network namespace to
host namespace and viceversa.

while it is true that it does not have the ability to choose the
source address, I mentioned there is a nat POSTROUTING rule that
changes the original source IP address into one assigned to traffic
emitted by the user id (the --direct rules I posted). maybe I am not
understanding the kernel flow when traffic is from a local ip to a
local ip, and also a nat involved? I admit it's not a common setup.
Just hoping for an elightenment, even if this is not firewalld's
fault.
> > Why does web can initiate outgoing to anywhere?
>
> Educated guess - because source address does not match your zone definition.

but the tcpdump shows the source address and the zone definition shows
the same address.

[Andrei Borzenkov]

On Sun, Oct 15, 2023 at 9:34 PM Mai Ling  wrote:
> On Sun, Oct 15, 2023 at 9:04 PM Andrei Borzenkov  wrote:
> > On 15.10.2023 20:31, Mai Ling wrote:
> > > Why does the response from api goes to destination being primary IP
> > >
> > > of the host, instead of the IP alias of the web container?
> >
> > Reply to TCP connection always goes to the source IP address. If
> > application did not explicitly select specific source IP when
> > establishing connection, kernel picks up one of suitable addresses. I
> > suppose kernel simply takes the first address assigned to the interface.
> >
> > I have no idea how traffic from containers is mapped to the available
> > addresses, but I am sure firewalld is not involved in this decision.
>
> in podman rootless, pasta is a process spawned when starting the
> container. it forwards packets from container network namespace to
> host namespace and viceversa.
>
> while it is true that it does not have the ability to choose the
> source address, I mentioned there is a nat POSTROUTING rule that
> changes the original source IP address into one assigned to traffic
> emitted by the user id (the --direct rules I posted). maybe I am not
> understanding the kernel flow when traffic is from a local ip to a
> local ip, and also a nat involved? I admit it's not a common setup.
> Just hoping for an elightenment, even if this is not firewalld's
> fault.

Maybe someone here can answer it, but it would be better to ask on
pasta support channels. It requires some knowledge of what this
program actually does with the packets it receives from containers. It
certainly cannot forward them unmodified.
> > > Why does web can initiate outgoing to anywhere?
> >
> > Educated guess - because source address does not match your zone definition.
>
> but the tcpdump shows the source address and the zone definition shows
> the same address.

From the host (and firewalld running on thi shost) point of view there
is no forwarding. All packets originated from the local host. The
rules for forwarding between two zones are never involved here. So the
only option I see is to use rich rules in HOST -> ANY policy (which
maps to OUTPUT hook).

[Mai Ling]

On Mon, Oct 16, 2023 at 12:17 PM Andrei Borzenkov  wrote:
> On Sun, Oct 15, 2023 at 9:34 PM Mai Ling  wrote:
> > On Sun, Oct 15, 2023 at 9:04 PM Andrei Borzenkov  wrote:
> > > On 15.10.2023 20:31, Mai Ling wrote:

[...]
> > > > Why does web can initiate outgoing to anywhere?
> > >
> > > Educated guess - because source address does not match your zone definition.
> >
> > but the tcpdump shows the source address and the zone definition shows
> > the same address.
>
> From the host (and firewalld running on thi shost) point of view there
> is no forwarding. All packets originated from the local host. The
> rules for forwarding between two zones are never involved here. So the
> only option I see is to use rich rules in HOST -> ANY policy (which
> maps to OUTPUT hook).

So I did create a policy, but traffic still passes. I then removed the
policy and created "--direct" rules: traffic does not pass, as
expected.
What am I doing wrong?

Pasted below all commands and output as reproductible steps on a
minimal installation of RHEL9:

[user@el9 ~]$ sudo nmcli connection modify enp3s0 +ipv4.addresses
192.168.1.22/26
[user@el9 ~]$ sudo nmcli connection modify enp3s0 +ipv4.addresses
192.168.1.44/26
[user@el9 ~]$ sudo nmcli connection modify enp3s0 +ipv4.addresses
192.168.1.55/26
[user@el9 ~]$ sudo nmcli device reapply enp3s0
Connection successfully reapplied to device 'enp3s0'.
[user@el9 ~]$ sudo firewall-cmd --get-active-zones
public
  interfaces: enp3s0
[user@el9 ~]$ sudo firewall-cmd --list-all --zone=public
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp3s0
  sources:
  services: cockpit dhcpv6-client ssh
  ports:
  protocols:
  forward: yes
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:
[user@el9 ~]$ sudo firewall-cmd --permanent --new-policy=output
success
[user@el9 ~]$ sudo firewall-cmd --permanent --policy=output --set-target=REJECT
success
[user@el9 ~]$ sudo firewall-cmd --permanent --policy=output
--add-rich-rule='rule family="ipv4" source address="192.168.1.22"
destination address="192.168.1.44" port port="8080" protocol="tcp"
accept'
success
[user@el9 ~]$ sudo firewall-cmd --permanent --policy=output
--add-ingress-zone=HOST
success
[user@el9 ~]$ sudo firewall-cmd --permanent --policy=output
--add-egress-zone=ANY
success
[user@el9 ~]$ sudo firewall-cmd --reload
success
[user@el9 ~]$ sudo firewall-cmd --info-policy=output
output (active)
  priority: -1
  target: REJECT
  ingress-zones: HOST
  egress-zones: ANY
  services:
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:
        rule family="ipv4" source address="192.168.1.22" destination
address="192.168.1.44" port port="8080" protocol="tcp" accept

[user@el9 ~]$ sudo nft list ruleset
table inet firewalld {
        chain mangle_PREROUTING {
                type filter hook prerouting priority mangle + 10; policy accept;
                jump mangle_PREROUTING_ZONES
        }

chain mangle_PREROUTING_POLICIES_pre {
                jump mangle_PRE_policy_allow-host-ipv6
        }

chain mangle_PREROUTING_ZONES {
                iifname "enp3s0" goto mangle_PRE_public
                goto mangle_PRE_public
        }

chain mangle_PREROUTING_POLICIES_post {
        }

chain nat_PREROUTING {
                type nat hook prerouting priority dstnat + 10; policy accept;
                jump nat_PREROUTING_ZONES
        }

chain nat_PREROUTING_POLICIES_pre {
                jump nat_PRE_policy_allow-host-ipv6
        }

chain nat_PREROUTING_ZONES {
                iifname "enp3s0" goto nat_PRE_public
                goto nat_PRE_public
        }

chain nat_PREROUTING_POLICIES_post {
        }

chain nat_POSTROUTING {
                type nat hook postrouting priority srcnat + 10; policy accept;
                jump nat_POSTROUTING_ZONES
        }

chain nat_POSTROUTING_POLICIES_pre {
        }

chain nat_POSTROUTING_ZONES {
                oifname "enp3s0" goto nat_POST_public
                goto nat_POST_public
        }

chain nat_POSTROUTING_POLICIES_post {
        }

chain nat_OUTPUT {
                type nat hook output priority -90; policy accept;
                jump nat_OUTPUT_POLICIES_pre
                jump nat_OUTPUT_POLICIES_post
        }

chain nat_OUTPUT_POLICIES_pre {
                jump nat_OUT_policy_output
        }

chain nat_OUTPUT_POLICIES_post {
        }

chain filter_PREROUTING {
                type filter hook prerouting priority filter + 10; policy accept;
                icmpv6 type { nd-router-advert, nd-neighbor-solicit } accept
                meta nfproto ipv6 fib saddr . mark . iif oif missing drop
        }

chain filter_INPUT {
                type filter hook input priority filter + 10; policy accept;
                ct state { established, related } accept
                ct status dnat accept
                ct state invalid drop
                iifname "lo" accept
                jump filter_INPUT_ZONES
                reject with icmpx admin-prohibited
        }

chain filter_FORWARD {
                type filter hook forward priority filter + 10; policy accept;
                ct state { established, related } accept
                ct status dnat accept
                ct state invalid drop
                iifname "lo" accept
                ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24,
2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28,
2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
                jump filter_FORWARD_ZONES
                reject with icmpx admin-prohibited
        }

chain filter_OUTPUT {
                type filter hook output priority filter + 10; policy accept;
                ct state { established, related } accept
                oifname "lo" accept
                ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24,
2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28,
2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
                jump filter_OUTPUT_POLICIES_pre
                jump filter_OUTPUT_POLICIES_post
        }

chain filter_INPUT_POLICIES_pre {
                jump filter_IN_policy_allow-host-ipv6
        }

chain filter_INPUT_ZONES {
                iifname "enp3s0" goto filter_IN_public
                goto filter_IN_public
        }

chain filter_INPUT_POLICIES_post {
        }

chain filter_FORWARD_POLICIES_pre {
        }

chain filter_FORWARD_ZONES {
                iifname "enp3s0" goto filter_FWD_public
                goto filter_FWD_public
        }

chain filter_FORWARD_POLICIES_post {
        }

chain filter_OUTPUT_POLICIES_pre {
                jump filter_OUT_policy_output
        }

chain filter_OUTPUT_POLICIES_post {
        }
...
[default filter IN stripped]
...
[default empty nat stripped]
...
[default empty FWD stripped]
...
[defaults empty nat and mangle stripped]
...
[default allow host ipv6 policy stripped]
...
        chain filter_OUT_policy_output {
                jump filter_OUT_policy_output_pre
                jump filter_OUT_policy_output_log
                jump filter_OUT_policy_output_deny
                jump filter_OUT_policy_output_allow
                jump filter_OUT_policy_output_post
                reject with icmpx admin-prohibited
        }

chain filter_OUT_policy_output_pre {
        }

chain filter_OUT_policy_output_log {
        }

chain filter_OUT_policy_output_deny {
        }

chain filter_OUT_policy_output_allow {
                ip daddr 192.168.1.44 ip saddr 192.168.1.22 tcp dport
8080 ct state { new, untracked } accept
        }

chain filter_OUT_policy_output_post {
        }

chain nat_OUT_policy_output {
                jump nat_OUT_policy_output_pre
                jump nat_OUT_policy_output_log
                jump nat_OUT_policy_output_deny
                jump nat_OUT_policy_output_allow
                jump nat_OUT_policy_output_post
        }

chain nat_OUT_policy_output_pre {
        }

chain nat_OUT_policy_output_log {
        }

chain nat_OUT_policy_output_deny {
        }

chain nat_OUT_policy_output_allow {
        }

chain nat_OUT_policy_output_post {
        }
}

[user@el9 ~]$ coproc sudo tcpdump -qnni any -s 1024 -w firewalld.pcap
[1] 15505
[user@el9 ~]$ tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2),
snapshot length 1024 bytes

[user@el9 ~]$ cat api.sh
while true; do   ncat -lvk 192.168.13.44 8080 -c '
  request=""
  while IFS= read -r line; do
    line=$(echo "$line" | tr -d "\r")
    if echo "$line" | grep -qE "^GET /" ; then
      request=$(echo "$line" | cut -d " " -f2)
    elif [ "x$line" = x ] ; then
      HTTP_RESPONSE="HTTP/1.1 200 OK\r\nContent-Type:
text/plain\r\n\r\nthis is 192.168.13.44 port 8080\r\n"
      echo -en "$HTTP_RESPONSE"
      break
    fi
  done'; done

[user@el9 ~]$ cat api2.sh
while true; do   ncat -lvk 192.168.13.55 8080 -c '
  request=""
  while IFS= read -r line; do
    line=$(echo "$line" | tr -d "\r")
    if echo "$line" | grep -qE "^GET /" ; then
      request=$(echo "$line" | cut -d " " -f2)
    elif [ "x$line" = x ] ; then
      HTTP_RESPONSE="HTTP/1.1 200 OK\r\nContent-Type:
text/plain\r\n\r\nthis is 192.168.13.55 port 8080\r\n"
      echo -en "$HTTP_RESPONSE"
      break
    fi
  done'; done

[user@el9 ~]$ http_proxy='' ALL_PROXY='' curl --interface 192.168.1.22
-m1 -v 192.168.1.55:8080
*   Trying 192.168.1.55:8080...
* Name '192.168.1.22' family 2 resolved to '192.168.1.22' family 2
* Local port: 0
* Connected to 192.168.1.55 (192.168.1.55) port 8080 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.55:8080
> User-Agent: curl/7.76.1
> Accept: */*

* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/plain
* no chunk, no close, no size. Assume close to signal end
<
this is 192.168.1.55 port 8080
* Closing connection 0
[user@el9 ~]$ http_proxy='' ALL_PROXY='' curl --interface 192.168.1.22
-m1 -v 192.168.1.44:8080
*   Trying 192.168.1.44:8080...
* Name '192.168.1.22' family 2 resolved to '192.168.1.22' family 2
* Local port: 0
* Connected to 192.168.1.44 (192.168.1.44) port 8080 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.44:8080
> User-Agent: curl/7.76.1
> Accept: */*

* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/plain
* no chunk, no close, no size. Assume close to signal end
<
this is 192.168.1.44 port 8080
* Closing connection 0

[user@el9 ~]$ tcpdump -qnnr firewalld.pcap
reading from file firewalld.pcap, link-type LINUX_SLL2 (Linux cooked
v2), snapshot length 1024
Warning: interface names might be incorrect
02:25:46.893238 enp3s0 M   IP 192.168.1.30.25353 > 224.0.0.249.25353:
UDP, length 41
02:25:46.893372 enp3s0 M   IP 192.168.1.30.25353 > 224.0.0.249.25353:
UDP, length 477
02:25:52.353101 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
02:25:52.353122 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 0
02:25:52.353138 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
02:25:52.353482 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 82
02:25:52.353492 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 0
02:25:52.379587 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 78
02:25:52.379615 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
02:25:52.379681 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 0
02:25:52.379837 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
02:25:52.379878 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 0
02:25:56.990959 lo    In  IP 192.168.1.22.33200 > 192.168.1.44.8080: tcp 0
02:25:56.991004 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.33200: tcp 0
02:25:56.991033 lo    In  IP 192.168.1.22.33200 > 192.168.1.44.8080: tcp 0
02:25:56.991124 lo    In  IP 192.168.1.22.33200 > 192.168.1.44.8080: tcp 82
02:25:56.991133 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.33200: tcp 0
02:25:57.017231 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.33200: tcp 78
02:25:57.017268 lo    In  IP 192.168.1.22.33200 > 192.168.1.44.8080: tcp 0
02:25:57.017511 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.33200: tcp 0
02:25:57.017605 lo    In  IP 192.168.1.22.33200 > 192.168.1.44.8080: tcp 0
02:25:57.017671 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.33200: tcp 0

[user@el9 ~]$ sudo firewall-cmd --direct --add-rule ipv4 filter OUTPUT
1000 -p tcp -s 192.168.1.22 -d 192.168.1.44 --dport 8080 -j ACCEPT
success
[user@el9 ~]$ sudo firewall-cmd --direct --add-rule ipv4 filter OUTPUT
1500 -s 192.168.1.22 -j LOG --log-prefix 'REJECT: ' --log-level 1
success
[user@el9 ~]$ sudo firewall-cmd --direct --add-rule ipv4 filter OUTPUT
2000 -s 192.168.1.22 -j REJECT
success

[user@el9 ~]$ sudo firewall-cmd --direct --get-all-rules
ipv4 filter OUTPUT 1000 -p tcp -s 192.168.1.22 -d 192.168.1.44 --dport
8080 -j ACCEPT
ipv4 filter OUTPUT 1500 -s 192.168.1.22 -j LOG --log-prefix 'REJECT: '
--log-level 1
ipv4 filter OUTPUT 2000 -s 192.168.1.22 -j REJECT

[user@el9 ~]$ sudo nft list ruleset
table ip filter {
        chain OUTPUT {
                type filter hook output priority filter; policy accept;
                ip saddr 192.168.1.22 ip daddr 192.168.1.44 tcp dport
8080 counter packets 0 bytes 0 accept
                ip saddr 192.168.1.22 counter packets 0 bytes 0 log
prefix "REJECT: " level alert
                ip saddr 192.168.1.22 counter packets 0 bytes 0 reject
        }
}
[...]

[user@el9 ~]$ http_proxy='' ALL_PROXY='' curl --interface 192.168.1.22
-m1 -v 192.168.1.44:8080
*   Trying 192.168.1.44:8080...
* Name '192.168.1.22' family 2 resolved to '192.168.1.22' family 2
* Local port: 0
* Connected to 192.168.1.44 (192.168.1.44) port 8080 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.44:8080
> User-Agent: curl/7.76.1
> Accept: */*

* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/plain
* no chunk, no close, no size. Assume close to signal end
<
this is 192.168.1.44 port 8080
* Closing connection 0
[user@el9 ~]$ http_proxy='' ALL_PROXY='' curl --interface 192.168.1.22
-m1 -v 192.168.1.55:8080
*   Trying 192.168.1.55:8080...
* Name '192.168.1.22' family 2 resolved to '192.168.1.22' family 2
* Local port: 0
* connect to 192.168.1.55 port 8080 failed: Connection refused
* Failed to connect to 192.168.1.55 port 8080: Connection refused
* Closing connection 0
curl: (7) Failed to connect to 192.168.1.55 port 8080: Connection refused

[user@el9 ~]$ tcpdump -qnnr firewalld.pcap
reading from file firewalld.pcap, link-type LINUX_SLL2 (Linux cooked
v2), snapshot length 1024
Warning: interface names might be incorrect
02:33:00.098612 lo    In  IP 192.168.1.22.38738 > 192.168.1.44.8080: tcp 0
02:33:00.098642 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.38738: tcp 0
02:33:00.098660 lo    In  IP 192.168.1.22.38738 > 192.168.1.44.8080: tcp 0
02:33:00.098727 lo    In  IP 192.168.1.22.38738 > 192.168.1.44.8080: tcp 82
02:33:00.098735 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.38738: tcp 0
02:33:00.125065 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.38738: tcp 78
02:33:00.125096 lo    In  IP 192.168.1.22.38738 > 192.168.1.44.8080: tcp 0
02:33:00.125185 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.38738: tcp 0
02:33:00.125338 lo    In  IP 192.168.1.22.38738 > 192.168.1.44.8080: tcp 0
02:33:00.125362 lo    In  IP 192.168.1.44.8080 > 192.168.1.22.38738: tcp 0
02:33:03.959440 lo    In  IP 192.168.1.55 > 192.168.1.22: ICMP
192.168.1.55 tcp port 8080 unreachable, length 68

[user@el9 ~]$ journalctl -kb|tail -2
Oct 20 02:28:50 el9 kernel: Warning: Deprecated Driver is detected:
nft_compat will not be maintained in a future major release and may be
disabled
Oct 20 02:33:03 el9 kernel: REJECT: IN= OUT=lo SRC=192.168.1.22
DST=192.168.1.55 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=20926 DF
PROTO=TCP SPT=40286 DPT=8080 WINDOW=65495 RES=0x00 SYN URGP=0

[Andrei Borzenkov]

On 20.10.2023 02:54, Mai Ling wrote:
> On Mon, Oct 16, 2023 at 12:17 PM Andrei Borzenkov  wrote:
> > On Sun, Oct 15, 2023 at 9:34 PM Mai Ling  wrote:
> > > On Sun, Oct 15, 2023 at 9:04 PM Andrei Borzenkov  wrote:
> > > > On 15.10.2023 20:31, Mai Ling wrote:
>
> [...]
> > > > > Why does web can initiate outgoing to anywhere?
> > > >
> > > > Educated guess - because source address does not match your zone definition.
> > >
> > > but the tcpdump shows the source address and the zone definition shows
> > > the same address.
> >
> > From the host (and firewalld running on thi shost) point of view there
> > is no forwarding. All packets originated from the local host. The
> > rules for forwarding between two zones are never involved here. So the
> > only option I see is to use rich rules in HOST -> ANY policy (which
> > maps to OUTPUT hook).
>
> So I did create a policy, but traffic still passes. I then removed the
> policy and created "--direct" rules: traffic does not pass, as
> expected.
> What am I doing wrong?

I probably misunderstood what you intended. I assumed you wanted to 
block traffic to/from external hosts.
...
> [user@el9 ~]$ http_proxy='' ALL_PROXY='' curl --interface 192.168.1.22
> -m1 -v 192.168.1.44:8080
> Trying 192.168.1.44:8080...
> Name '192.168.1.22' family 2 resolved to '192.168.1.22' family 2
> Local port: 0
> Connected to 192.168.1.44 (192.168.1.44) port 8080 (#0)
> > GET / HTTP/1.1
> > Host: 192.168.1.44:8080
> > User-Agent: curl/7.76.1
> > Accept: */*
> Mark bundle as not supporting multiuse
>
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> no chunk, no close, no size. Assume close to signal end
>
> <
> this is 192.168.1.44 port 8080
> Closing connection 0
>
> [user@el9 ~]$ tcpdump -qnnr firewalld.pcap
> reading from file firewalld.pcap, link-type LINUX_SLL2 (Linux cooked
> v2), snapshot length 1024
> Warning: interface names might be incorrect
> 02:25:46.893238 enp3s0 M   IP 192.168.1.30.25353 > 224.0.0.249.25353:
> UDP, length 41
> 02:25:46.893372 enp3s0 M   IP 192.168.1.30.25353 > 224.0.0.249.25353:
> UDP, length 477
> 02:25:52.353101 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
> 02:25:52.353122 lo    In  IP 192.168.1.55.8080 > 192.168.1.22.59448: tcp 0
> 02:25:52.353138 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 0
> 02:25:52.353482 lo    In  IP 192.168.1.22.59448 > 192.168.1.55.8080: tcp 82

...

All your traffic is on loopback interface. If you look at rules 
generated by firewalld, it shortcuts loopback and always allows traffic 
here. So it never comes to checking other rules you added. When you use 
direct rules, you add *another* netfilter hook that runs *in addition* 
to nftables rules generated by firewalld, and both must allow the 
packet. That explains why adding direct rules works.

I do not think firewalld can be used to control traffic between 
different processes on the host itself without some serious redesign.