https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Bug ID: 1144100 Summary: rabbitmq restarts fail randomly Product: Fedora EPEL Version: epel7 Component: rabbitmq-server Assignee: lemenkov@gmail.com Reporter: bnemec@redhat.com QA Contact: extras-qa@fedoraproject.org CC: bnemec@redhat.com, erlang@lists.fedoraproject.org, extras-qa@fedoraproject.org, hubert.plociniczak@gmail.com, imcleod@redhat.com, jeckersb@redhat.com, josh@fornwall.com, lemenkov@gmail.com, lnie@redhat.com, ohadlevy@redhat.com, rjones@redhat.com, s@shk.io Depends On: 1059028
+++ This bug was initially created as a clone of Bug #1059028 +++
This still appears to be a problem in the EPEL version of rabbitmq-server.
Description of problem:
I originally encountered this issue when installing OpenStack via devstack but have since been able to reproduce it by simply executing commands inside of a fresh F20 install.
Version-Release number of selected component (if applicable):
rabbitmq-server-3.1.5-1.fc20.noarch
How reproducible:
Occurs regularly but not 100% of the time
Steps to Reproduce:
/sbin/service rabbitmq-server stop /sbin/service rabbitmq-server start rabbitmqctl change_password guest newpassword
Actual results:
About half the time, on a freshly installed F20, this will fail, claiming the node cannot be contacted. A typical error message:
[root@cob-dell5 ~]# rabbitmqctl change_password guest ozrootpw Changing password for user "guest" ... Error: unable to connect to node 'rabbit@cob-dell5': nodedown
DIAGNOSTICS ===========
nodes in question: ['rabbit@cob-dell5']
hosts, their running nodes and ports: - cob-dell5: [{rabbitmqctl2648,48609}]
current node details: - node name: 'rabbitmqctl2648@cob-dell5' - home dir: /var/lib/rabbitmq - cookie hash: 8DNoVu56TqDYWypW7YXDJw==
Expected results:
Changing password for user "guest" ... ...done.
--- Additional comment from Ian McLeod on 2014-01-28 21:36:45 EST ---
See the following thread on rdo-list for another example of this issue:
https://www.redhat.com/archives/rdo-list/2013-December/msg00058.html
As well as this more recent one, indicating tha the problem persists:
https://www.redhat.com/archives/rdo-list/2014-January/msg00164.html
--- Additional comment from Ian McLeod on 2014-02-27 17:51:21 EST ---
Note to anyone following. The devstack installer has a commit that attempts to workaround this issue by retrying the rabbitmq restart:
https://github.com/openstack-dev/devstack/commit/f6c001faf6ac5728e07c6bbdaa6...
Still, it would be nice to sort this out on the rabbitmq side eventually.
--- Additional comment from Fedora Admin XMLRPC Client on 2014-07-01 13:58:36 EDT ---
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
--- Additional comment from Fedora Update System on 2014-08-10 04:14:16 EDT ---
rabbitmq-server-3.1.5-9.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/rabbitmq-server-3.1.5-9.fc20
--- Additional comment from lnie on 2014-08-11 03:00:36 EDT ---
rabbitmq-server-3.1.5-9.fc20 works
--- Additional comment from Fedora Update System on 2014-08-14 22:35:38 EDT ---
Package rabbitmq-server-3.1.5-9.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing rabbitmq-server-3.1.5-9.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-9337/rabbitmq-server-3.1... then log in and leave karma (feedback).
--- Additional comment from Fedora Update System on 2014-08-15 20:28:02 EDT ---
rabbitmq-server-3.1.5-9.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1059028 [Bug 1059028] rabbitmq restarts fail randomly
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Charles Crouch ccrouch@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Severity|unspecified |high
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
John Eckersberg jeckersb@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|lemenkov@gmail.com |jeckersb@redhat.com
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Nathan G. Grennan redhat-bugzilla@cygnusx-1.org changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |redhat-bugzilla@cygnusx-1.o | |rg
--- Comment #1 from Nathan G. Grennan redhat-bugzilla@cygnusx-1.org --- I just ran into this issue. RHEL 7 Openstack has upgraded to 3.3.5-2. I recompiled the rabbitmq-server 3.3.5-2 fc22 src.rpm from koji, and it fixed my issue.
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Ian Wienand iwienand@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |iwienand@redhat.com
--- Comment #2 from Ian Wienand iwienand@redhat.com --- This is being seen in upstream testing for openstack
--- 2014-12-15 19:16:20.015 | ++ sudo rabbitmqctl list_users 2014-12-15 19:16:20.239 | Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown 2014-12-15 19:16:20.249 | 2014-12-15 19:16:20.249 | DIAGNOSTICS 2014-12-15 19:16:20.249 | =========== 2014-12-15 19:16:20.249 | 2014-12-15 19:16:20.249 | nodes in question: ['rabbit@devstack-centos7-rax-iad-100675'] 2014-12-15 19:16:20.250 | 2014-12-15 19:16:20.250 | hosts, their running nodes and ports: 2014-12-15 19:16:20.250 | - devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}] 2014-12-15 19:16:20.250 | 2014-12-15 19:16:20.250 | current node details: 2014-12-15 19:16:20.250 | - node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675' 2014-12-15 19:16:20.250 | - home dir: /var/lib/rabbitmq 2014-12-15 19:16:20.250 | - cookie hash: KieJnx1pnllKbHVihGcDqA== 2014-12-15 19:16:20.250 | 2014-12-15 19:16:20.256 | + out='Listing users ...' 2014-12-15 19:16:20.256 | + echo 'failed to list users' ---
This happens somewhat frequently on our centos 7 jobs (can search on [1] to see examples)
[1] http://logstash.openstack.org
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |MODIFIED
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
--- Comment #3 from Fedora Update System updates@fedoraproject.org --- rabbitmq-server-3.3.5-3.el7 has been submitted as an update for Fedora EPEL 7. https://admin.fedoraproject.org/updates/rabbitmq-server-3.3.5-3.el7
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|MODIFIED |ON_QA
--- Comment #4 from Fedora Update System updates@fedoraproject.org --- Package rabbitmq-server-3.3.5-3.el7: * should fix your issue, * was pushed to the Fedora EPEL 7 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing rabbitmq-server-3.3.5-3.el7' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-4742/rabbitmq-serve... then log in and leave karma (feedback).
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
--- Comment #5 from Fedora Update System updates@fedoraproject.org --- rabbitmq-server-3.3.5-4.el7 has been submitted as an update for Fedora EPEL 7. https://admin.fedoraproject.org/updates/rabbitmq-server-3.3.5-4.el7
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|ON_QA |CLOSED Fixed In Version| |rabbitmq-server-3.3.5-4.el7 Resolution|--- |ERRATA Last Closed| |2015-01-05 21:09:08
--- Comment #6 from Fedora Update System updates@fedoraproject.org --- rabbitmq-server-3.3.5-4.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
https://bugzilla.redhat.com/show_bug.cgi?id=1144100
David Hill dhill@cloudops.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |dhill@cloudops.com
--- Comment #7 from David Hill dhill@cloudops.com --- This created bugs in Ubuntu Vivid.
See : https://review.openstack.org/#/c/191090/
erlang@lists.fedoraproject.org