Recovering from Erlang issue

[lt56]

We are running RabbitMQ 3.6.6, erlang version Erlang/OTP 17 [erts-6.4]. I start rabbitmq as a service on my CentOS Linux system. I get no errors at startup. At least none that I am aware of. 

RabbitMQ seems to be running successfully. I can post to exchanges and queues from NiFi. I can get into Rabbit through it's Management Console.

I've detected a problem for several reasons:

1. When I try to add virtual hosts, exchanges, or new users through the Management Console none of my changes show up. A month or two ago, they would. I added an exchange and a dozen queues, for example.

2. When I use rabbitmqctl to add a new user, I get this error:

"Erlang VM I/O system is damaged, restart needed"

This is accompanied by a few more lines of somewhat generic, somewhat standard looking erlang dump messages.

I hope folks who have wrestled with this can offer some guidance to a few questions:

1. how does one avoid this erlang state in the future? I have done some research in the open source and find that a few folks have reported this error, but the solution paths and the causes are somewhat poorly understood.

2. Given that I start rabbitmq as a service, I want to be sure that I ask before I attempt to directly shutdown and restart rabbitmq or erlang so that I don't do it in a bad sequence that leaves me hung up. Should I stop the service, then run 'rabbitmqctl stop' in an effort to stop rabbitmq and erlang? Should I avoid using init:stop() at the erl prompt, or can rabbitmq survive a direct restart of the erlang foundation?

3. What happens if I continue to get these erlang errors when I try to stop rabbitmq? Must I kill -9 PIDs, and reinstall erlang? Does reinstalling erlang mean I must reinstall rabbitmq? I sure hop enot.

4. I notice in the nohup.out file created by my /etc/init.d/rabbitmq script at startup, the broker starts and there is a message at end of that short file "Gracefully halting Erlang VM". Is that normal? I would guess that Rabbit would want Erlang to stay up, bu tperhaps that doesn't necessarily mean the full Erlang VM.  

Thanks for any help.   -Jim

[Michael Klishin]

1. If this is the message introduced in https://github.com/rabbitmq/rabbitmq-server/pull/747 (which is my best guess), the solution is to upgrade to a newer Erlang version (3.6.6 supports everything up to 19.3.x)

2. If you stop the service it should take care of stopping the node. Your problem is likely the I/O handling issue that breaks CLI tools.

3. Again, the issue probably comes down to 1. Reinstalling Erlang means reinstalling Erlang, not both Erlang and RabbitMQ (at least I don't see a reason to do that)

4. Why would RabbitMQ leave the Erlang VM running? The only outcome of that is confusion. "Gracefully halting Erlang VM" is a message expected from RabbitMQ when it shuts down after being requested to do so.

And remember that the standalone OS X package bundles an Erlang/OTP runtime, so it won't use

any other OTP version you have installed. Other package just use what's available.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[James McMahon]

Thank you very much Michael. I will work on updating my Erlang version.

One follow-up question: when you say this...

"And remember that the standalone OS X package bundles an Erlang/OTP runtime, so it won't use any other OTP version you have installed"

...it seems to contradict the first directive on this page, Install a recent version of Erlang.

https://www.rabbitmq.com/install-generic-unix.html

Why is that? I take what you mentioned to mean RabbitMQ comes with erlang already integrated, no need to install it separately. Yet this install guide at the Rabbit site seems to contradict that.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/GvN4tYA-Q-A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

[Michael Klishin]

That page describes the generic UNIX package, not the standalone OS X one:
http://www.rabbitmq.com/install-standalone-mac.html.

Only the standalone OS X package bundles Erlang. Which is what I believe my
earlier email said. 

On 26 April 2017 at 19:07:16, James McMahon (jsmcm...@gmail.com) wrote:
> Thank you very much Michael. I will work on updating my Erlang version.
>
> One follow-up question: when you say this...
> "And remember that the standalone OS X package bundles an Erlang/OTP
> runtime, so it won't use any other OTP version you have installed"
> ...it seems to contradict the first directive on this page, Install a
> recent version of Erlang.
> https://www.rabbitmq.com/install-generic-unix.html
> Why is that? I take what you mentioned to mean RabbitMQ comes with erlang
> already integrated, no need to install it separately. Yet this install
> guide at the Rabbit site seems to contradict that.
>
> On Wed, Apr 26, 2017 at 12:17 PM, Michael Klishin

> >> email to rabbitmq-user...@googlegroups.com.

> >> To post to this group, send email to rabbitm...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
> >
> > --
> > MK
> >
> > Staff Software Engineer, Pivotal/RabbitMQ
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "rabbitmq-users" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/
> > topic/rabbitmq-users/GvN4tYA-Q-A/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to

> > rabbitmq-user...@googlegroups.com.

> > To post to this group, send email to rabbitm...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --

> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To post to this group, send an email to rabbitm...@googlegroups.com.