RabbitMQ stuck in reboot loop

[Thomas Speckert]

Hi,

We are experiencing a strange issue on a server where RabbitMQ/Erlang is stuck in a reboot loop. RabbitMQ appears to crash on start, and then Erlang is restarted. This happens endlessly in a loop (in the windows event log, there are warnings from ErlSrv every 10 seconds: Erlang service restarted and Restarted erlang machine.

Details over our setup:

• This a Windows server
• Only a single RabbitMQ node
• RabbitMQ is natively installed, using a custom install process. The Erlang and RabbitMQ/Erlang files are essentially just copied onto the machine.
• We run RabbitMQ as a Windows Service
• Install data (erlang, rabbitmq) is stored in one folder. Runtime data (i.e. the rabbitMQ database) is stored in a second folder.

We use RabbitMQ as part of a larger software suite. The problem is triggered when we upgrade our software. What happens:

• The RabbitMQ service is stopped and uninstalled
• (new) We kill any remaining erl/epmd/erlsvr processes
• All installation data is removed  (runtime data is not removed)
• The installation from the new version is installed (in this case, there is actually no change to the RabbitMQ files as we haven't upgraded the version)
• The RabbitMQ service is installed and started

At this point we get stuck in the reboot loop. We are able to fix it by manually stopping the RabbitMQ service, uninstalling it, removing all runtime data (it's transient anyways), and then finally reinstalling and starting the Rabbit MQ service.

Based on the logs (attached), it appears that some file is corrupted somewhere, but I'm not sure what and I'm stuck as to how we can fix this issue permanently.

If anyone has any suggestions, or can identify any clear causes from the logs it would be greatly appreciated.

[Michal Kuratczyk]

This is a very old, unsupported, RabbitMQ version. We are not going to spend time investigating it.

At first glance, some files are missing, so I guess your upgrade process is wrong.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/e5ae6847-b143-471c-95dc-a6024144abacn%40googlegroups.com.

--

Michał

RabbitMQ team

[Thomas Speckert]

Thank you Michal for the response. 

Yes unfortunately we still have this version in use on some systems, but we do have the possibility to update to a recent version. If the issue persists after an update I'll follow up. 

We've had issues in the past with the update process, namely due to having problems stopping the RabbitMQ Windows Service. Even after stopping and uninstalling the windows service, it seems that some erlang processes sometimes still hang around for awhile, which has caused us issues. We do kill them now, but that doesn't seem to have helped here. Were there any know improvements/fixes in recent versions which could help?

[Michal Kuratczyk]

I'm not aware of any fixes in this area but I've also never heard of this problem so maybe it's related to even older versions.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/5977083e-621e-4bb4-924e-5c1c178b0ddan%40googlegroups.com.

--

Michał

RabbitMQ team