7:02 a.m.
Hi,
I'm trying to kill some instances and in the dashboard, but they're
stuck with "deleting" status.
In compute.log I can see this: http://fpaste.org/An9x/
The VMs themselves are up and running..
openstack-status says it's fine (i don't use swift or volumes)
openstack-status
== Nova services ==
openstack-nova-api: active
openstack-nova-cert: active
openstack-nova-compute: active
openstack-nova-network: active
openstack-nova-scheduler: active
openstack-nova-volume: inactive (disabled on boot)
== Glance services ==
openstack-glance-api: active
openstack-glance-registry: active
== Keystone service ==
openstack-keystone: active
== Horizon service ==
openstack-dashboard: active
== Quantum services ==
quantum-server: inactive (disabled on boot)
== Swift services ==
openstack-swift-proxy: inactive (disabled on boot)
openstack-swift-account: inactive (disabled on boot)
openstack-swift-container: inactive (disabled on boot)
openstack-swift-object: inactive (disabled on boot)
== Cinder services ==
openstack-cinder-api: active
openstack-cinder-scheduler: active
openstack-cinder-volume: active
== Support services ==
mysqld: active
httpd: active
libvirtd: active
tgtd: inactive (disabled on boot)
qpidd: active
memcached: active
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
8:27 a.m.
On 14.01.2013 13:02, Nux! wrote:
Hi,
I'm trying to kill some instances and in the dashboard, but they're
stuck with "deleting" status.
In compute.log I can see this: http://fpaste.org/An9x/
The VMs themselves are up and running..
I wonder if it's not something to do with the recent reinstall of
horizon I had to do.
Upon restarting compute I noticed:
2013-01-14 16:24:14 1611 INFO nova.compute.manager [-] [instance:
ba2e36c6-8795-4e87-863a-34d2bc4b0f45] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:14 1611 INFO nova.compute.manager [-] [instance:
1f4d96c3-dda4-438b-9cb9-2f84945301c5] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:17 1611 INFO nova.compute.manager [-] [instance:
6ef6682a-7004-426c-80cf-988928bc1cb7] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:17 1611 INFO nova.compute.manager [-] [instance:
9fd1ec97-480f-42f7-ae7d-b0f690d15d03] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:17 1611 INFO nova.compute.manager [-] [instance:
d227b1ea-8715-476b-9c29-434d8c8c4139] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:17 1611 INFO nova.compute.manager [-] [instance:
d9c866cc-088e-4485-88f7-690a525484ca] During sync_power_state the
instance has a pending task. Skip.
2013-01-14 16:24:18 1611 INFO nova.compute.manager [-] [instance:
6a0803f3-2bb0-4d14-851c-94b8eabafb85] During sync_power_state the
instance has a pending task. Skip.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
8:57 a.m.
On 14.01.2013 14:27, Nux! wrote:
On 14.01.2013 13:02, Nux! wrote:
> Hi,
>
> I'm trying to kill some instances and in the dashboard, but they're
> stuck with "deleting" status.
> In compute.log I can see this: http://fpaste.org/An9x/
>
> The VMs themselves are up and running..
I wonder if it's not something to do with the recent reinstall of
horizon I had to do.
Might be the horizon after all, this is what error_log shows:
[Mon Jan 14 14:54:13 2013] [error] endpoint_type='adminURL')
[Mon Jan 14 14:54:13 2013] [error] File
"/usr/lib/python2.6/site-packages/keystoneclient/service_catalog.py",
line 73, in url_for
[Mon Jan 14 14:54:13 2013] [error] raise
exceptions.EndpointNotFound('Endpoint not found.')
[Mon Jan 14 14:54:13 2013] [error] EndpointNotFound: Endpoint not
found.
[Mon Jan 14 14:54:13 2013] [error] unable to retrieve service catalog
with token
[Mon Jan 14 14:54:13 2013] [error] Traceback (most recent call last):
[Mon Jan 14 14:54:13 2013] [error] File
"/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line
135, in _extract_service_catalog
[Mon Jan 14 14:54:13 2013] [error] endpoint_type='adminURL')
[Mon Jan 14 14:54:13 2013] [error] File
"/usr/lib/python2.6/site-packages/keystoneclient/service_catalog.py",
line 73, in url_for
[Mon Jan 14 14:54:13 2013] [error] raise
exceptions.EndpointNotFound('Endpoint not found.')
[Mon Jan 14 14:54:13 2013] [error] EndpointNotFound: Endpoint not
found.
[Mon Jan 14 14:54:14 2013] [error] unable to retrieve service catalog
with token
[Mon Jan 14 14:54:14 2013] [error] Traceback (most recent call last):
[Mon Jan 14 14:54:14 2013] [error] File
"/usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py", line
135, in _extract_service_catalog
[Mon Jan 14 14:54:14 2013] [error] endpoint_type='adminURL')
[Mon Jan 14 14:54:14 2013] [error] File
"/usr/lib/python2.6/site-packages/keystoneclient/service_catalog.py",
line 73, in url_for
[Mon Jan 14 14:54:14 2013] [error] raise
exceptions.EndpointNotFound('Endpoint not found.')
[Mon Jan 14 14:54:14 2013] [error] EndpointNotFound: Endpoint not
found.
What does that mean?
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
10:11 a.m.
On 14.01.2013 14:57, Nux! wrote:
On 14.01.2013 14:27, Nux! wrote:
> On 14.01.2013 13:02, Nux! wrote:
>> Hi,
>>
>> I'm trying to kill some instances and in the dashboard, but they're
>> stuck with "deleting" status.
>> In compute.log I can see this: http://fpaste.org/An9x/
>>
>> The VMs themselves are up and running..
Deletion from command line doesn't work either.
Funny thing is that if I create a new vm it works great, deletion too.
Just these VMs seem stuck.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
10:15 a.m.
On 14.01.2013 16:11, Nux! wrote:
On 14.01.2013 14:57, Nux! wrote:
> On 14.01.2013 14:27, Nux! wrote:
>> On 14.01.2013 13:02, Nux! wrote:
>>> Hi,
>>>
>>> I'm trying to kill some instances and in the dashboard, but
>>> they're
>>> stuck with "deleting" status.
>>> In compute.log I can see this: http://fpaste.org/An9x/
>>>
>>> The VMs themselves are up and running..
Deletion from command line doesn't work either.
Funny thing is that if I create a new vm it works great, deletion
too. Just these VMs seem stuck.
I'm trying this method
http://docs.openstack.org/trunk/openstack-compute/admin/content/reset-sta...
but what I get is:
ERROR: Policy doesn't allow compute_extension:admin_actions:resetState
to be performed. (HTTP 403) (Request-ID:
req-7b46cdd9-5f11-48ba-b8a7-eda0882588bc)
Anything?
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
11:14 a.m.
Hi, I've had something similar and eventually the instance terminated by itself, where
you running something with RDS connection by any chance?what was the instance doing when
you've terminated? \\sse1st... p.s. how's life in the Delta quadrant?
Date: Mon, 14 Jan 2013 16:15:15 +0000
From: nux(a)li.nux.ro
To: cloud(a)lists.fedoraproject.org
Subject: Re: instances refuse to terminate
On 14.01.2013 16:11, Nux! wrote:
> On 14.01.2013 14:57, Nux! wrote:
>> On 14.01.2013 14:27, Nux! wrote:
>>> On 14.01.2013 13:02, Nux! wrote:
>>>> Hi,
>>>>
>>>> I'm trying to kill some instances and in the dashboard, but
>>>> they're
>>>> stuck with "deleting" status.
>>>> In compute.log I can see this: http://fpaste.org/An9x/
>>>>
>>>> The VMs themselves are up and running..
>
> Deletion from command line doesn't work either.
> Funny thing is that if I create a new vm it works great, deletion
> too. Just these VMs seem stuck.
I'm trying this method
http://docs.openstack.org/trunk/openstack-compute/admin/content/reset-sta...
but what I get is:
ERROR: Policy doesn't allow compute_extension:admin_actions:resetState
to be performed. (HTTP 403) (Request-ID:
req-7b46cdd9-5f11-48ba-b8a7-eda0882588bc)
Anything?
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
_______________________________________________
cloud mailing list
cloud(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
11:18 a.m.
On 14.01.2013 17:14, Shahar Ettinger wrote:
Hi, I've had something similar and eventually the instance
terminated
by itself, where you running something with RDS connection by any
chance?what was the instance doing when you've terminated?
\\sse1st... p.s. how's life in the Delta quadrant?
What is RDS?
Well, I could wait for stuff to eventually time out, but who knows if
or when it happens. Right know these machines use valuable resources
that I can't use for something else.
I've opened a thread on the openstack mailing list about manually
killing instances, let's see what they say. I'm comfortable with libvirt
side of things, but not with the internals of openstack.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
2 p.m.
On 14.01.2013 13:02, Nux! wrote:
Hi,
I'm trying to kill some instances and in the dashboard, but they're
stuck with "deleting" status.
In compute.log I can see this: http://fpaste.org/An9x/
I ended up wiping them from DB with the help of this guy's howto
http://www.sebastien-han.fr/blog/2012/10/08/delete-a-vm-in-error-state-fo...
Then killed the VMs themselves with virsh destro & undefine.
I guess I'll never find out what the heck caused all this mayhem..
Unless it happens again. :)
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
4118
days inactive
4118
days old
7 comments
2 participants
participants (2)
-
nux＠li.nux.ro -
Shahar Ettinger