virt-manager destroying the guest leaves a "ghost" in xend.

[Konrad Rzeszutek Wilk]

I tried to install a guest and midway during it decided to install something
else so I stopped the process and killed the guest.

But got an error and xend.log has this:

[2011-10-10 17:42:06 961] DEBUG (XendDomainInfo:1276) XendDomainInfo.destroyDevice:
deviceClass = vbd, device = vbd/51712
[2011-10-10 17:43:29 961] ERROR (SrvBase:88) Request destroy failed.
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xen/web/SrvBase.py", line 85, in
perform
    return op_method(op, req)
  File "/usr/lib64/python2.7/site-packages/xen/xend/server/SrvDomain.py", line
89, in op_destroy
    return self.xd.domain_destroy(self.dom.domid)
  File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomain.py", line 1315,
in domain_destroy
    dominfo = self.domain_lookup_nr(domid)
  File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomain.py", line 588, in
domain_lookup_nr
    if int(domid) in self.domains:
TypeError: int() argument must be a string or a number, not 'NoneType'

and "xm list" tells me

[root@tst006 /]# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  3343     2     r-----    162.2
F15_test                                        1024     1                 0.0


Quitting virt-manager and then 'Deleting' the F15_test makes xend finally remove
the
ghost:

[2011-10-10 17:45:27 961] INFO (XendDomain:1126) Domain F15_test
(18eaec13-8ed3-ad0c-6473-7947366fce02) deleted.


So.. has anybody else hit this?

[Marko Ristola]

Hi Konrad.

I can easilly repeat a following situation:
1. Start up a guest on Virt-Manager. The quest is created via virt-manager, Fedora 14, as
a network install.
2. Shut it down via Virt-Manager's menus.
3. Wait until Xen reports on xend.log:
"Domain has shutdown: name=fedora14 id=2 reason=poweroff"
On virt-manager, guest's status remains "Running".
If I try any operations for the guest, I get a Python exception,
something like you found, but related to the operation.

4. Disconnect from localhost(xen) on Virt-Manager.
5. Connect into localhost(xen) on Virt-Manager.
6. Now the guest status of fedora14 is "Shutoff".
7. Now we can go back to phase 1 and start again.

So the bug is (on high level) something like: Xen turns off the guest, but virt-manager
won't
get notified, or won't detect, about the state change of Xen quest:
"Running" -> "Shutoff".

So my hypothesis on your problem is, that Xen already turned guest's power off,
and virt-manager didn't notice it. So virt-manager asks Xen to do commands which
aren't
supported for turned off quest.

Regards,
Marko Ristola

On 10/11/2011 01:07 AM, Konrad Rzeszutek Wilk wrote:
>  I tried to install a guest and midway during it decided to install
> something
>  else so I stopped the process and killed the guest.
>
>  But got an error and xend.log has this:
>
>  [2011-10-10 17:42:06 961] DEBUG (XendDomainInfo:1276) XendDomainInfo.destroyDevice:
> deviceClass = vbd, device = vbd/51712
>  [2011-10-10 17:43:29 961] ERROR (SrvBase:88) Request destroy failed.
>  Traceback (most recent call last):
>     File "/usr/lib64/python2.7/site-packages/xen/web/SrvBase.py", line 85, in
> perform
>       return op_method(op, req)
>     File "/usr/lib64/python2.7/site-packages/xen/xend/server/SrvDomain.py", line
> 89, in op_destroy
>       return self.xd.domain_destroy(self.dom.domid)
>     File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomain.py", line 1315,
> in domain_destroy
>       dominfo = self.domain_lookup_nr(domid)
>     File "/usr/lib64/python2.7/site-packages/xen/xend/XendDomain.py", line 588,
> in domain_lookup_nr
>       if int(domid) in self.domains:
>  TypeError: int() argument must be a string or a number, not 'NoneType'
>
>  and "xm list" tells me
>
>  [root@tst006 /]# xm list
>  Name                                        ID   Mem VCPUs      State   Time(s)
>  Domain-0                                     0  3343     2     r-----    162.2
>  F15_test                                        1024     1                 0.0
>
>
>  Quitting virt-manager and then 'Deleting' the F15_test makes xend finally remove
> the
>  ghost:
>
>  [2011-10-10 17:45:27 961] INFO (XendDomain:1126) Domain F15_test
> (18eaec13-8ed3-ad0c-6473-7947366fce02) deleted.
>
>
>  So.. has anybody else hit this?
>  --
>  xen mailing list
>  xen(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/xen

[Tim Flink]

On Fri, 14 Oct 2011 21:04:17 +0300
Marko Ristola <marko.ristola(a)kolumbus.fi&gt; wrote:

>  
>  Hi Konrad.
>  
>  I can easilly repeat a following situation:
>  1. Start up a guest on Virt-Manager. The quest is created via
>  virt-manager, Fedora 14, as a network install. 2. Shut it down via
>  Virt-Manager's menus. 3. Wait until Xen reports on xend.log:
>  "Domain has shutdown: name=fedora14 id=2 reason=poweroff"
>  On virt-manager, guest's status remains "Running".
>  If I try any operations for the guest, I get a Python exception,
>  something like you found, but related to the operation.
>  
>  4. Disconnect from localhost(xen) on Virt-Manager.
>  5. Connect into localhost(xen) on Virt-Manager.
>  6. Now the guest status of fedora14 is "Shutoff".
>  7. Now we can go back to phase 1 and start again.
>  
>  So the bug is (on high level) something like: Xen turns off the
>  guest, but virt-manager won't get notified, or won't detect, about
>  the state change of Xen quest: "Running" -> "Shutoff".
>  
>  So my hypothesis on your problem is, that Xen already turned guest's
>  power off, and virt-manager didn't notice it. So virt-manager asks
>  Xen to do commands which aren't supported for turned off quest.
>  
>  Regards,
>  Marko Ristola 
I really need to be better about communicating the xen related bugs
that I'm filing. Konrad's initial question sounded slightly different
but I never got around to testing with the method that he listed.

Part of this sounds like a virt-manager bug that I filed last week:
 * 746007 - Status of Xen guests is not updated in virt-manager [1]

Interesting note:
 - In this case, status is correct for 'virsh list --all' and 'xm list'
   even though virt-manager isn't showing DomU status correctly

Tim

[1] https://bugzilla.redhat.com/show_bug.cgi?id=746007