Hi,
I have a workstation with Fedora 16 using NetworkManager getting a static IP address via DHCP from a central DHCP server. I have a couple of VMs on that workstation that use a routed network device in libvirt that I would also like to acquire their IP address from the central DHCP server.
I set up the virtual routed network device virbr1 and configured a VM (with CentOS 6) to use it. When the VM starts I see in Wireshark the DHCP broadcasts on the virbr1 interface but those broadcasts are not seen on the p21p1 (the old eth0) interface on the workstation and definitely don't make it to the central DHCP server. I guess I may need some additional IPTables rules to forward the VMs DHCP requests to the central DHCP server? Does anyone know what IPTables rule(s) I should add to make this work?
Here is an overview of configs. Apologies for the iptables linewrap. I don't know how to force Thunderbird not to do that but I also put the info up at pastebin: http://pastebin.com/3u6cVUux
Virtual Network Device "vmr": <network> <name>vmr</name> <forward mode='route'/> <bridge name='virbr1' /> <ip address='192.168.198.1' netmask='255.255.255.0'> </ip> </network>
IP_forward is enabled: # sysctl net.ipv4.ip_forward net.ipv4.ip_forward = 1
# virsh net-list --all Name State Autostart ----------------------------------------- vmr active yes default inactive yes
# route -n Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.0.0.138 0.0.0.0 UG 0 0 0 p21p1 10.0.0.0 0.0.0.0 255.255.255.0 U 1 0 0 p21p1 192.168.198.0 192.168.198.1 255.255.255.0 UG 0 0 0 virbr1 192.168.198.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr1
# ifconfig
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:5372 errors:0 dropped:0 overruns:0 frame:0 TX packets:5372 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1652676 (1.5 MiB) TX bytes:1652676 (1.5 MiB)
p21p1 Link encap:Ethernet HWaddr DE:AD:BE:EF:DE:AD inet addr:10.0.0.135 Bcast:10.0.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5433191 errors:0 dropped:0 overruns:0 frame:0 TX packets:2791523 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7957954289 (7.4 GiB) TX bytes:219450376 (209.2 MiB) Interrupt:47 Base address:0xe000
virbr1 Link encap:Ethernet HWaddr 52:54:00:4B:6B:C9 inet addr:192.168.198.1 Bcast:192.168.198.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:51 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10635 (10.3 KiB) TX bytes:0 (0.0 b)
vnet0 Link encap:Ethernet HWaddr FE:54:C6:00:64:01 inet6 addr: fe80::fc54:c6ff:fe00:6401/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:51 errors:0 dropped:0 overruns:0 frame:0 TX packets:1272 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:11349 (11.0 KiB) TX bytes:66888 (65.3 KiB)
The output of iptables -v -n -L is also available at: http://pastebin.com/3u6cVUux
# iptables -v -n -L Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- virbr1 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- virbr1 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 27 9099 ACCEPT udp -- virbr1 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- virbr1 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 5176K 7687M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 18 1200 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 57 8880 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22 3081 520K REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- * virbr1 0.0.0.0/0 192.168.198.0/24 0 0 ACCEPT all -- virbr1 * 192.168.198.0/24 0.0.0.0/0 0 0 ACCEPT all -- virbr1 virbr1 0.0.0.0/0 0.0.0.0/0 0 0 REJECT all -- * virbr1 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 0 0 REJECT all -- virbr1 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT 17791 packets, 1525K bytes) pkts bytes target prot opt in out source destination
Thanks!
Regards, Patrick
On 11/28/2011 12:07 PM, Patrick Lists wrote:
Hi,
I have a workstation with Fedora 16 using NetworkManager getting a static IP address via DHCP from a central DHCP server. I have a couple of VMs on that workstation that use a routed network device in libvirt that I would also like to acquire their IP address from the central DHCP server.
I set up the virtual routed network device virbr1 and configured a VM (with CentOS 6) to use it. When the VM starts I see in Wireshark the DHCP broadcasts on the virbr1 interface but those broadcasts are not seen on the p21p1 (the old eth0) interface on the workstation and definitely don't make it to the central DHCP server. I guess I may need some additional IPTables rules to forward the VMs DHCP requests to the central DHCP server? Does anyone know what IPTables rule(s) I should add to make this work?
My understanding and experience is that you really want to bridge the virtual machines to get them on the real network.
It can get very interesting when you add VLANs per virtual machine, or groups of virtual machines.
NetworkManager can handle bridges ok, and I do so on my desktop, but on all of our VM servers we shut of/don't install NetworkManager and just use network.
The principles are the same which ever you wish to use.
# yum -y install bridge-utils
This is a blip from our kickstart post-install for desktops and 'livecd' based VM servers:
# set up a bridge on eht0
cat > /etc/sysconfig/network-scripts/ifcfg-br0 <<_EOF DEVICE=br0 ONBOOT=yes TYPE=Bridge BOOTPROTO=dhcp STP=off DELAY=0 NM_CONTROLLED="yes" _EOF
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 <<_EOF DEVICE=eth0 ONBOOT=yes BRIDGE=br0 NM_CONTROLLED="yes" _EOF
This gives you the bridge, and all the virt tools will see it. You would use p21p1, of course, or force udev to give you the old names.
Now that you have the bridge, you can specify it in the installs of your VMs.
They can do dhcp on the same network the server does.
This can be extended without much fuss to include VLANs as well. Here is a sample from a startup script for a livecd based VM server that requires a different VLAN for one of the two VMs he serves:
modprobe 8021q vconfig add eth1 840 vconfig add eth1 60 ifconfig inet 0.0.0.0 eth1.60
cat > /etc/sysconfig/network-scripts/ifcfg-br1 <<_EOF DEVICE=br1 ONBOOT=yes TYPE=Bridge BOOTPROTO=none STP=off DELAY=0 _EOF
cat > /etc/sysconfig/network-scripts/ifcfg-eth1.60 <<_EOF DEVICE=eth1.60 ONBOOT=yes VLAN=yes BRIDGE=br1 _EOF
cat > /etc/sysconfig/network-scripts/ifcfg-eth1.840 <<_EOF DEVICE=eth1.840 ONBOOT=yes VLAN=yes IPADDR=10.150.0.102 NETMASK=255.255.252.0 _EOF
ifup eth1.840 ifup eth1.60 ifup br1
It is my experience that when using tagged VLANs such as this, that the bridge cannot successfully do dhcp because the underlying VLAN does not come up until after the bridge in this case. Other than that one CAVEAT, this works well.
Need a VM on VLAN 60? Just define his interface on bridge br1. No need to do any bridging or VLAN setup inside the VM.
Bridges are the bomb. Use 'em.
Good luck!
-
Emilio Lopez -
Patrick Lists -
Phil Meyer