9:22 p.m.
Hi,
I'm running with fc5, kernel-xen0-2.6.17-1.2174.
Yesterday I had a working Xen installation with a domU also running FC5,
same kernel, udev, xen, libvirt, and same hardware.
Following a rebuild of the host and reinstall/update of FC5 to same
update levels, I'm now hitting a problem installing a domU, where it
appears its not setting up a bridged interface for the domU. I think it
might be hotplug related?
I've got the same udev, and its running, and I can see the rules files
in /etc/udev/rules.d.
Below is the output of several logs and commands.. as far as I can see,
the network-bridge script is good, and its set up my interfaces
correctly. One minor complication is that my bridged ethernet card is
on device eth2, but I've got the amendment in xend-config.sxp to call
"(network-script 'network-bridge netdev=eth2')".
I checked the kernel-xen list in bugzilla but couldn't find anything
there on hotplug problems.
Any help in figuring out the root cause would be much appreciated!
Having gone from working to not-working in a reinstall of the same setup
has me very confused.
#### brctl show ####
bridge name bridge id STP enabled interfaces
xenbr2 8000.feffffffffff no peth2
vif0.2
#### ifconfig ####
eth2 Link encap:Ethernet HWaddr 00:14:6C:8C:2E:25
inet addr:192.168.0.10 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::214:6cff:fe8c:2e25/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:353 errors:0 dropped:0 overruns:0 frame:0
TX packets:396 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:42427 (41.4 KiB) TX bytes:97320 (95.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:660 errors:0 dropped:0 overruns:0 frame:0
TX packets:660 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3375221 (3.2 MiB) TX bytes:3375221 (3.2 MiB)
peth2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:362 errors:0 dropped:0 overruns:0 frame:0
TX packets:457 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:43683 (42.6 KiB) TX bytes:104598 (102.1 KiB)
Interrupt:18
vif0.2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:396 errors:0 dropped:0 overruns:0 frame:0
TX packets:354 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:97320 (95.0 KiB) TX bytes:42517 (41.5 KiB)
xenbr2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:62 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:9920 (9.6 KiB) TX bytes:468 (468.0 b)
#### /var/log/xen-hotplug.log ####
interface vif1.0 does not exist!
#### /var/log/xend.log ####
[2006-08-30 11:17:25 xend] INFO (SrvDaemon:283) Xend Daemon started
[2006-08-30 11:17:25 xend] INFO (SrvDaemon:287) Xend changeset:
unavailable .
[2006-08-30 11:17:25 xend.XendDomainInfo] DEBUG (XendDomainInfo:207)
XendDomainInfo.recreate({'paused': 0, 'cpu_time': 10867218609L,
'ssidref': 0, 'handle': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0], 'shutdown_reason': 0, 'dying': 0, 'dom': 0, 'mem_kb':
961860,
'maxmem_kb': -4, 'max_vcpu_id': 0, 'crashed': 0,
'running': 1,
'shutdown': 0, 'online_vcpus': 1, 'blocked': 0})
[2006-08-30 11:17:25 xend.XendDomainInfo] INFO (XendDomainInfo:219)
Recreating domain 0, UUID 00000000-0000-0000-0000-000000000000.
[2006-08-30 11:17:25 xend.XendDomainInfo] WARNING (XendDomainInfo:241)
No vm path in store for existing domain 0
[2006-08-30 11:17:25 xend.XendDomainInfo] DEBUG (XendDomainInfo:701)
Storing VM details: {'name': 'Domain-0', 'on_reboot':
'restart',
'on_poweroff': 'destroy', 'uuid':
'00000000-0000-0000-0000-000000000000', 'on_crash': 'restart',
'vcpus':
'1', 'vcpu_avail': '1', 'memory': '940',
'maxmem': '940'}
[2006-08-30 11:17:25 xend.XendDomainInfo] DEBUG (XendDomainInfo:736)
Storing domain details: {'cpu/0/availability': 'online',
'memory/target': '962560', 'name': 'Domain-0',
'console/limit':
'1048576', 'vm': '/vm/00000000-0000-0000-0000-000000000000',
'domid': '0'}
[2006-08-30 11:17:25 xend] DEBUG (XendDomain:153) number of vcpus to use
is 0
[2006-08-30 11:17:25 xend.XendDomainInfo] DEBUG (XendDomainInfo:919)
XendDomainInfo.handleShutdownWatch
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:185)
XendDomainInfo.create(['vm', ['name', 'xen-www'],
['memory', '256'],
['maxmem', '256'], ['vcpus', '1'], ['on_poweroff',
'destroy'],
['on_reboot', 'destroy'], ['on_crash', 'destroy'],
['image', ['linux',
['kernel', '/var/lib/xen/vmlinuz.1WDQ9c'], ['ramdisk',
'/var/lib/xen/initrd.img.jBbqHv'], ['args', ' root=/dev/xvd
method=ftp://virtual/pub ']]], ['device', ['vbd', ['dev',
'/dev/xvda'],
['uname', 'file:/dev/raid/xen_www'], ['mode', 'w']]],
['device', ['vif',
['mac', '00:16:3e:11:48:67'], ['bridge', 'xenbr0'],
['script',
'/etc/xen/scripts/vif-bridge']]]])
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:291)
parseConfig: config is ['vm', ['name', 'xen-www'],
['memory', '256'],
['maxmem', '256'], ['vcpus', '1'], ['on_poweroff',
'destroy'],
['on_reboot', 'destroy'], ['on_crash', 'destroy'],
['image', ['linux',
['kernel', '/var/lib/xen/vmlinuz.1WDQ9c'], ['ramdisk',
'/var/lib/xen/initrd.img.jBbqHv'], ['args', ' root=/dev/xvd
method=ftp://virtual/pub ']]], ['device', ['vbd', ['dev',
'/dev/xvda'],
['uname', 'file:/dev/raid/xen_www'], ['mode', 'w']]],
['device', ['vif',
['mac', '00:16:3e:11:48:67'], ['bridge', 'xenbr0'],
['script',
'/etc/xen/scripts/vif-bridge']]]]
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:390)
parseConfig: result is {'uuid': None, 'on_crash': 'destroy',
'on_reboot': 'destroy', 'image': ['linux',
['kernel',
'/var/lib/xen/vmlinuz.1WDQ9c'], ['ramdisk',
'/var/lib/xen/initrd.img.jBbqHv'], ['args', ' root=/dev/xvd
method=ftp://virtual/pub ']], 'on_poweroff': 'destroy',
'bootloader_args': None, 'cpus': None, 'name': 'xen-www',
'backend': [],
'vcpus': 1, 'cpu_weight': None, 'features': None,
'vcpu_avail': None,
'memory': 256, 'device': [('vbd', ['vbd', ['dev',
'/dev/xvda'],
['uname', 'file:/dev/raid/xen_www'], ['mode', 'w']]),
('vif', ['vif',
['mac', '00:16:3e:11:48:67'], ['bridge', 'xenbr0'],
['script',
'/etc/xen/scripts/vif-bridge']])], 'bootloader': None, 'cpu':
None,
'maxmem': 256}
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:1216)
XendDomainInfo.construct: None
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:1248)
XendDomainInfo.initDomain: 1 1.0
[2006-08-30 11:22:23 xend] DEBUG (balloon:134) Balloon: free 62; need
257; retries: 10.
[2006-08-30 11:22:23 xend] DEBUG (balloon:143) Balloon: setting dom0
target to 745.
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:987)
Setting memory target of domain Domain-0 (0) to 745 MiB.
[2006-08-30 11:22:23 xend] DEBUG (balloon:128) Balloon: free 257; need
257; done.
[2006-08-30 11:22:23 xend] INFO (image:134) buildDomain os=linux dom=1
vcpus=1
[2006-08-30 11:22:23 xend] DEBUG (image:177) dom = 1
[2006-08-30 11:22:23 xend] DEBUG (image:178) image =
/var/lib/xen/vmlinuz.1WDQ9c
[2006-08-30 11:22:23 xend] DEBUG (image:179) store_evtchn = 1
[2006-08-30 11:22:23 xend] DEBUG (image:180) console_evtchn = 2
[2006-08-30 11:22:23 xend] DEBUG (image:181) cmdline =
root=/dev/xvd method=ftp://virtual/pub
[2006-08-30 11:22:23 xend] DEBUG (image:182) ramdisk =
/var/lib/xen/initrd.img.jBbqHv
[2006-08-30 11:22:23 xend] DEBUG (image:183) vcpus = 1
[2006-08-30 11:22:23 xend] DEBUG (image:184) features =
[2006-08-30 11:22:23 xend] DEBUG (blkif:24) exception looking up device
number for /dev/xvda: [Errno 2] No such file or directory: '/dev/xvda'
[2006-08-30 11:22:23 xend] DEBUG (DevController:110) DevController:
writing {'virtual-device': '51712', 'backend-id': '0',
'state': '1',
'backend': '/local/domain/0/backend/vbd/1/51712'} to
/local/domain/1/device/vbd/51712.
[2006-08-30 11:22:23 xend] DEBUG (DevController:112) DevController:
writing {'domain': 'xen-www', 'frontend':
'/local/domain/1/device/vbd/51712', 'dev': '/dev/xvda',
'state': '1',
'params': '/dev/raid/xen_www', 'mode': 'w',
'frontend-id': '1', 'type':
'file'} to /local/domain/0/backend/vbd/1/51712.
[2006-08-30 11:22:23 xend] DEBUG (DevController:110) DevController:
writing {'backend-id': '0', 'mac': '00:16:3e:11:48:67',
'handle': '0',
'state': '1', 'backend':
'/local/domain/0/backend/vif/1/0'} to
/local/domain/1/device/vif/0.
[2006-08-30 11:22:23 xend] DEBUG (DevController:112) DevController:
writing {'bridge': 'xenbr0', 'domain': 'xen-www',
'handle': '0',
'script': '/etc/xen/scripts/vif-bridge', 'state': '1',
'frontend':
'/local/domain/1/device/vif/0', 'mac': '00:16:3e:11:48:67',
'frontend-id': '1'} to /local/domain/0/backend/vif/1/0.
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:701)
Storing VM details: {'uuid': '5b6415d8-f3a6-d64b-67c8-d8f7afb98e95',
'on_reboot': 'destroy', 'start_time': '1156900943.67',
'on_poweroff':
'destroy', 'name': 'xen-www', 'vcpus': '1',
'vcpu_avail': '1', 'memory':
'256', 'on_crash': 'destroy', 'image': "(linux
(kernel
/var/lib/xen/vmlinuz.1WDQ9c) (ramdisk /var/lib/xen/initrd.img.jBbqHv)
(args ' root=/dev/xvd method=ftp://virtual/pub '))", 'maxmem':
'256'}
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:736)
Storing domain details: {'console/ring-ref': '237071',
'console/port':
'2', 'name': 'xen-www', 'console/limit':
'1048576', 'vm':
'/vm/5b6415d8-f3a6-d64b-67c8-d8f7afb98e95', 'domid': '1',
'cpu/0/availability': 'online', 'memory/target': '262144',
'store/ring-ref': '237072', 'store/port': '1'}
[2006-08-30 11:22:23 xend.XendDomainInfo] DEBUG (XendDomainInfo:919)
XendDomainInfo.handleShutdownWatch
[2006-08-30 11:22:23 xend] DEBUG (DevController:143) Waiting for devices
vif.
[2006-08-30 11:22:23 xend] DEBUG (DevController:149) Waiting for 0.
[2006-08-30 11:22:23 xend] DEBUG (DevController:449)
hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2006-08-30 11:22:23 xend] DEBUG (DevController:449)
hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2006-08-30 11:22:23 xend] DEBUG (DevController:463)
hotplugStatusCallback 2.
[2006-08-30 11:22:23 xend] ERROR (SrvBase:87) Request wait_for_devices
failed.
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/xen/web/SrvBase.py", line 85,
in perform
return op_method(op, req)
File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomain.py",
line 72, in op_wait_for_devices
return self.dom.waitForDevices()
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py",
line 1511, in waitForDevices
self.waitForDevices_(c)
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py",
line 1063, in waitForDevices_
return self.getDeviceController(deviceClass).waitForDevices()
File
"/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py",
line 145, in waitForDevices
return map(self.waitForDevice, self.deviceIDs())
File
"/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py",
line 161, in waitForDevice
raise VmError("Device %s (%s) could not be connected. "
VmError: Device 0 (vif) could not be connected. Backend device not found.
[2006-08-30 11:22:24 xend] INFO (XendDomain:365) Domain xen-www (1)
unpaused.
#### /var/log/messages ####
Aug 30 11:17:26 virtual kernel: Bridge firewalling registered
Aug 30 11:17:26 virtual kernel: device vif0.2 entered promiscuous mode
Aug 30 11:17:26 virtual NET[2453]: /sbin/dhclient-script : updated
/etc/resolv.conf
Aug 30 11:17:26 virtual kernel: audit(1156900646.201:3): dev=vif0.2
prom=256 old_prom=0 auid=4294967295
Aug 30 11:17:26 virtual kernel: xenbr2: port 1(vif0.2) entering learning
state
Aug 30 11:17:26 virtual dhclient: bound to 192.168.0.10 -- renewal in
43167 seconds.
Aug 30 11:17:26 virtual kernel: xenbr2: topology change detected,
propagating
Aug 30 11:17:26 virtual kernel: xenbr2: port 1(vif0.2) entering
forwarding state
Aug 30 11:17:26 virtual kernel: r8169: peth2: link up
Aug 30 11:17:26 virtual kernel: device peth2 entered promiscuous mode
Aug 30 11:17:26 virtual kernel: audit(1156900646.249:4): dev=peth2
prom=256 old_prom=0 auid=4294967295
Aug 30 11:17:26 virtual kernel: xenbr2: port 2(peth2) entering learning
state
Aug 30 11:17:26 virtual kernel: xenbr2: topology change detected,
propagating
Aug 30 11:17:26 virtual kernel: xenbr2: port 2(peth2) entering
forwarding state
Aug 30 11:20:37 virtual ntpd[2004]: synchronized to LOCAL(0), stratum 10
Aug 30 11:20:37 virtual ntpd[2004]: kernel time sync disabled 0041
Aug 30 11:21:40 virtual ntpd[2004]: kernel time sync enabled 0001
Aug 30 11:22:23 virtual logger: /etc/xen/scripts/vif-bridge: brctl addif
xenbr0 vif1.0 failed
Aug 30 11:22:46 virtual ntpd[2004]: synchronized to 192.231.203.132,
stratum 2