libvirtd and public access to guests

[Pavel Lisy]

Hello

I've started playing with libvirt and I have question?

What is proper way to make guest accessible from net.

I have mode=nat /var/lib/libvirt/network/default.xml.

libvirtd makes this rules in FORWARD chain

-A FORWARD -d 192.168.231.0/24 -o virbr0 -m state --state
RELATED,ESTABLISHED -j ACCEPT 
-A FORWARD -s 192.168.231.0/24 -i virbr0 -j ACCEPT 
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

If I add 
iptables -I FORWARD -i eth0 -o virbr0 -j ACCEPT
guests are accessible

My question is:
Is is possible write this somewhere to configuration?

I've tried to put it in /etc/sysconfig/iptables but it libvirtd put his
rules before mine.

I've found two directories
/var/lib/libvirt/iptables/filter
/var/lib/libvirt/iptables/nat

I suppose I can write my rules here but I haven't find any docs about
format. Can somebody help me with it?

Pavel

[Richard W.M. Jones]

On Thu, Oct 08, 2009 at 01:09:35PM +0200, Pavel Lisy wrote:
> I've started playing with libvirt and I have question?

This question is probably better asked on libvir-list.

https://www.redhat.com/mailman/listinfo/libvir-list
> What is proper way to make guest accessible from net.
>
> I have mode=nat /var/lib/libvirt/network/default.xml.
>
> libvirtd makes this rules in FORWARD chain
>
> -A FORWARD -d 192.168.231.0/24 -o virbr0 -m state --state
> RELATED,ESTABLISHED -j ACCEPT 
> -A FORWARD -s 192.168.231.0/24 -i virbr0 -j ACCEPT 
> -A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
> -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
> -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
> -A FORWARD -j REJECT --reject-with icmp-host-prohibited
>
> If I add 
> iptables -I FORWARD -i eth0 -o virbr0 -j ACCEPT
> guests are accessible

An alternate way is to create your own bridge (however you want to
configure it), then make it a network that guests can see and connect
to, using commands like 'virsh net-create', 'virsh net-dumpxml' and
'virsh net-edit'.

The XML format is described here:

http://libvirt.org/formatnetwork.html
> My question is:
> Is is possible write this somewhere to configuration?
>
> I've tried to put it in /etc/sysconfig/iptables but it libvirtd put his
> rules before mine.

I think libvirtd will trash your virbr0 definitions, so maybe setting
up your own bridge is a better idea.

Rich.

[Daniel P. Berrange]

On Thu, Oct 08, 2009 at 01:09:35PM +0200, Pavel Lisy wrote:
> Hello
>
> I've started playing with libvirt and I have question?
>
> What is proper way to make guest accessible from net.

The shared physical device, bridging option is what you want
to use

http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
> I have mode=nat /var/lib/libvirt/network/default.xml.

NAT is for outbound internet access only - it doesn't allow
for remote clients to connect to your VM.
> libvirtd makes this rules in FORWARD chain
>
> -A FORWARD -d 192.168.231.0/24 -o virbr0 -m state --state
> RELATED,ESTABLISHED -j ACCEPT 
> -A FORWARD -s 192.168.231.0/24 -i virbr0 -j ACCEPT 
> -A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
> -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
> -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
> -A FORWARD -j REJECT --reject-with icmp-host-prohibited
>
> If I add 
> iptables -I FORWARD -i eth0 -o virbr0 -j ACCEPT
> guests are accessible
>
> My question is:
> Is is possible write this somewhere to configuration?
>
> I've tried to put it in /etc/sysconfig/iptables but it libvirtd put his
> rules before mine.
>
> I've found two directories
> /var/lib/libvirt/iptables/filter
> /var/lib/libvirt/iptables/nat
>
> I suppose I can write my rules here but I haven't find any docs about
> format. Can somebody help me with it?

You shouldn't try to overwrite/override libvirt's rules here, since libvirt
will likely just break your changes at some point. You really want to switch
to a bridged network config, instead of the NAT based one

Daniel

[Pavel Lisy]

Daniel P. Berrange píše v Čt 08. 10. 2009 v 21:57 +0100:
> On Thu, Oct 08, 2009 at 01:09:35PM +0200, Pavel Lisy wrote:
> > Hello
> >
> > I've started playing with libvirt and I have question?
> >
> > What is proper way to make guest accessible from net.
>
> The shared physical device, bridging option is what you want
> to use
>
> http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
> > I have mode=nat /var/lib/libvirt/network/default.xml.
>
> NAT is for outbound internet access only - it doesn't allow
> for remote clients to connect to your VM.
> > libvirtd makes this rules in FORWARD chain
> >
> > -A FORWARD -d 192.168.231.0/24 -o virbr0 -m state --state
> > RELATED,ESTABLISHED -j ACCEPT 
> > -A FORWARD -s 192.168.231.0/24 -i virbr0 -j ACCEPT 
> > -A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
> > -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
> > -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
> > -A FORWARD -j REJECT --reject-with icmp-host-prohibited
> >
> > If I add 
> > iptables -I FORWARD -i eth0 -o virbr0 -j ACCEPT
> > guests are accessible
> >
> > My question is:
> > Is is possible write this somewhere to configuration?
> >
> > I've tried to put it in /etc/sysconfig/iptables but it libvirtd put his
> > rules before mine.
> >
> > I've found two directories
> > /var/lib/libvirt/iptables/filter
> > /var/lib/libvirt/iptables/nat
> >
> > I suppose I can write my rules here but I haven't find any docs about
> > format. Can somebody help me with it?
>
> You shouldn't try to overwrite/override libvirt's rules here, since libvirt
> will likely just break your changes at some point. You really want to switch
> to a bridged network config, instead of the NAT based one

I've tried it but it isn't what I want. I don't want to have guests in
the our LAN network.

I want to test LDAP replication and samba config for two different
offices so I want to make separated networks accessible from our LAN.

I've tried routed network before. I've made necessary changes in
configuration on our router.

<network> 
  <name>routed</name>
  <uuid>fe53ef22-ae5b-47c6-ba24-fe21ea3e06a3</uuid>
  <forward mode='route' dev='eth0' />
  <bridge name='virbr0' stp='on' forwardDelay='0' />
  <ip address='192.168.231.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.231.100' end='192.168.231.254' />
      <host mac='54:52:00:6a:25:73' name='ldap1.virt-hk.tmapy.cz' ip='192.168.231.41' />
      <host mac='54:52:00:2b:b9:03' name='ldap2.virt-hk.tmapy.cz' ip='192.168.231.42' />
    </dhcp>
  </ip>
</network>

It was working but libvirt couldn't give IP addresses to guests through
dhcpmasq. Is it normal?

Questions:

1. Can I make network in mode='route' and use dhcp for guests? How?

2. NAT is working perfectly to my needs - routing, dhcp, ...
   (see my config below)
   Is it possible put extra iptables rules to libvirt configuration?

With <forward mode='nat'/> dhcp worked but I had to change few 
      IP tables rules to make this net accessible:

in file default.xml:
<network>
  <name>default</name>
  <uuid>fe53ef22-ae5b-47c6-ba24-fe21ea3e06a3</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' forwardDelay='0' />
  <ip address='192.168.231.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.231.100' end='192.168.231.254' />
      <host mac='54:52:00:6a:25:73' name='ldap1.virt-hk.tmapy.cz' ip='192.168.231.41' />
      <host mac='54:52:00:2b:b9:03' name='ldap2.virt-hk.tmapy.cz' ip='192.168.231.42' />
    </dhcp>
  </ip>
</network>

iptables changes:
# remove masquerading 
iptables -t nat -D POSTROUTING -s 192.168.231.0/24 ! -d 192.168.231.0/24 -j MASQUERADE
# open virtnet from eth0
iptables -I FORWARD -i eth0 -o virbr0 -j ACCEPT
iptables -D FORWARD -d 192.168.231.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT

Pavel