How to set up firewalld for a routing bridge (brouter) in Fedora Server 32?

[Peter Boy]

I’m working on a configuration to make multiple VMs accessible via public IPv4 and IPv6
addresses. The Internet provider routes all addresses (IPv4 and IPv6 subnets) to the MAC
address of the physical adapter. So I have to route to the VMs on the server (Fedora
Server 32). The more common bridge solution for direct VM access doesn’t work. I have
configured the physical interface (enp3s0) accordingly and attached a virtual bridge
(vbr3s0) to which the VMs connect. In addition, there is a libvirt bridge for private
communication between VM's.  


Everything works great as long as firewalld is *stopped*. 


The goal is to configure the host firewall so that everything is forwarded by default and
the VM's firewall controls the specific access. At the same time, it should be
possible to restrict access for individual IPs to certain ports (these IPs provide public
access for containers instead of VMs).


Unfortunately, I'm not that familiar with firewalld. I found the following solution
options

(a)
Place the bridge (vbr3s0) in a zone providing target=ACCEPT, e.g. trusted or nm-shared.
In this way, routing works for IPv6, but not for IPv4. 

(b)
Defining "direct rules" for forwarding. Various posts all boiled down to: 

[...]# firewall-cmd --direct  --get-all-rules 
ipv4 filter FORWARD 0 -i enp3s0 -o vbr3s0 -j ACCEPT
ipv4 filter FORWARD 0 -i vbr3s0 -o enp3s0 -j ACCEPT

Unfortunately, IPv4 still does not work with these rules either. Obviously, I have missed
something. 



Unfortunately I am not getting any further and would be very appreciative of any help.


Thanks
Peter





My IP configuration

2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group
default qlen 1000
    link/ether 30:85:a9:ee:23:d4 brd ff:ff:ff:ff:ff:ff
    inet 144.76.7.86/32 scope global noprefixroute enp3s0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f8:190:8255::2/128 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::3285:a9ff:feee:23d4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: vbr3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group
default qlen 1000
    link/ether 8e:5d:9f:58:45:4b brd ff:ff:ff:ff:ff:ff
    inet 144.76.7.86/32 scope global noprefixroute vbr3s0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f8:190:8255::2/64 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::8c5d:9fff:fe58:454b/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group
default qlen 1000
    link/ether 52:54:00:c9:33:cc brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state
DOWN group default qlen 1000
    link/ether 52:54:00:c9:33:cc brd ff:ff:ff:ff:ff:ff
6: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vbr3s0
state UNKNOWN group default qlen 1000
    link/ether fe:54:00:80:ea:aa brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe80:eaaa/64 scope link 
       valid_lft forever preferred_lft forever
7: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0
state UNKNOWN group default qlen 1000
    link/ether fe:54:00:7f:46:50 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe7f:4650/64 scope link 
       valid_lft forever preferred_lft forever


My zones so far:

[...]# firewall-cmd --get-active-zones
FedoraServer
  interfaces: enp3s0
libvirt
  interfaces: virbr0
trusted
  interfaces: vbr3s0

[Carlos Vidal]

Based on what you describe, it seems that IPv6 is not being filtered at
all. Regarding IPv4, I guess you need to add also INPUT rules, not just
forwarding.

It seems that firewalld is segmenting your network zones blocking traffic
where it should not. Maybe it is overkill to use firewalld, as you will not
be modifying your rules dynamically. When I started to use firewalld I
found it easier to write scripts using iptables, and once the firewall was
running fine, I translated the rules to XML in firewalld.

I also find it easier to edit the /etc/firewalld/zones/*.xml files directly
rather than using the CLI, then reload with "firewall-cmd --reload".
Typically "INPUT" rules will be in the "public.xml" (or Fedora*.xml)
file,
and forward and NAT rules in /etc/firewalld/direct.xml. You can also add
comments in this way (via <!-- ... -->).

In your case, if the only thing you want is to control incoming IPs, it may
be enough to run a small script with iptables than to deploy the full
firewalld with a configuration that will disable most of it anyway.

Carlos

On Sat, Jan 2, 2021 at 12:03 PM Peter Boy <pboy(a)uni-bremen.de&gt; wrote:

>  I’m working on a configuration to make multiple VMs accessible via
> public
>  IPv4 and IPv6 addresses. The Internet provider routes all addresses (IPv4
>  and IPv6 subnets) to the MAC address of the physical adapter. So I have to
>  route to the VMs on the server (Fedora Server 32). The more common bridge
>  solution for direct VM access doesn’t work. I have configured the physical
>  interface (enp3s0) accordingly and attached a virtual bridge (vbr3s0) to
>  which the VMs connect. In addition, there is a libvirt bridge for private
>  communication between VM's.
>
>
>  Everything works great as long as firewalld is *stopped*.
>
>
>  The goal is to configure the host firewall so that everything is forwarded
>  by default and the VM's firewall controls the specific access. At the same
>  time, it should be possible to restrict access for individual IPs to
>  certain ports (these IPs provide public access for containers instead of
>  VMs).
>
>
>  Unfortunately, I'm not that familiar with firewalld. I found the following
>  solution options
>
>  (a)
>  Place the bridge (vbr3s0) in a zone providing target=ACCEPT, e.g. trusted
>  or nm-shared.
>  In this way, routing works for IPv6, but not for IPv4.
>
>  (b)
>  Defining "direct rules" for forwarding. Various posts all boiled down to:
>
>  [...]# firewall-cmd --direct  --get-all-rules
>  ipv4 filter FORWARD 0 -i enp3s0 -o vbr3s0 -j ACCEPT
>  ipv4 filter FORWARD 0 -i vbr3s0 -o enp3s0 -j ACCEPT
>
>  Unfortunately, IPv4 still does not work with these rules either.
>  Obviously, I have missed something.
>
>
>
>  Unfortunately I am not getting any further and would be very appreciative
>  of any help.
>
>
>  Thanks
>  Peter
>
>
>
>
>
>  My IP configuration
>
>  2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state
>  UP group default qlen 1000
>      link/ether 30:85:a9:ee:23:d4 brd ff:ff:ff:ff:ff:ff
>      inet 144.76.7.86/32 scope global noprefixroute enp3s0
>         valid_lft forever preferred_lft forever
>      inet6 2a01:4f8:190:8255::2/128 scope global noprefixroute
>         valid_lft forever preferred_lft forever
>      inet6 fe80::3285:a9ff:feee:23d4/64 scope link noprefixroute
>         valid_lft forever preferred_lft forever
>  3: vbr3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
>  UP group default qlen 1000
>      link/ether 8e:5d:9f:58:45:4b brd ff:ff:ff:ff:ff:ff
>      inet 144.76.7.86/32 scope global noprefixroute vbr3s0
>         valid_lft forever preferred_lft forever
>      inet6 2a01:4f8:190:8255::2/64 scope global noprefixroute
>         valid_lft forever preferred_lft forever
>      inet6 fe80::8c5d:9fff:fe58:454b/64 scope link noprefixroute
>         valid_lft forever preferred_lft forever
>  4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
>  UP group default qlen 1000
>      link/ether 52:54:00:c9:33:cc brd ff:ff:ff:ff:ff:ff
>      inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
>         valid_lft forever preferred_lft forever
>  5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0
>  state DOWN group default qlen 1000
>      link/ether 52:54:00:c9:33:cc brd ff:ff:ff:ff:ff:ff
>  6: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master
>  vbr3s0 state UNKNOWN group default qlen 1000
>      link/ether fe:54:00:80:ea:aa brd ff:ff:ff:ff:ff:ff
>      inet6 fe80::fc54:ff:fe80:eaaa/64 scope link
>         valid_lft forever preferred_lft forever
>  7: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master
>  virbr0 state UNKNOWN group default qlen 1000
>      link/ether fe:54:00:7f:46:50 brd ff:ff:ff:ff:ff:ff
>      inet6 fe80::fc54:ff:fe7f:4650/64 scope link
>         valid_lft forever preferred_lft forever
>
>
>  My zones so far:
>
>  [...]# firewall-cmd --get-active-zones
>  FedoraServer
>    interfaces: enp3s0
>  libvirt
>    interfaces: virbr0
>  trusted
>    interfaces: vbr3s0
>  _______________________________________________
>  server mailing list -- server(a)lists.fedoraproject.org
>  To unsubscribe send an email to server-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>