Thread Dump and Heap Dump for RabbitMQ Server

302 views
Skip to first unread message

Arpan Gupta

unread,
Feb 23, 2018, 12:23:42 AM2/23/18
to rabbitmq-users

s it possible to get Thread Dump and Heap Dump in case of a RabbitMQ Server?


To be Specific: 

The RabbitMQ server I am referring to runs on a docker and I want to get Thread Dump and Heap Dump in such a case which would be useful in disaster analysis or other preventive alerts.

Luke Bakken

unread,
Feb 23, 2018, 10:40:24 AM2/23/18
to rabbitmq-users
Hi Arpan,

If you're talking about a core file, you should do some research into what other people do with docker containers. A quick search turned this up, for instance - https://github.com/moby/moby/issues/11740

There are articles online with information to do when you have an Erlang VM core file - https://www.erlang-solutions.com/blog/how-to-analyse-a-beam-core-dump.html

Finally, the RabbitMQ server log files should be the first place you check if you run into issues.

Thanks,
Luke
--
Staff Software Engineer
Pivotal / RabbitMQ

Michael Klishin

unread,
Feb 23, 2018, 11:36:24 AM2/23/18
to rabbitm...@googlegroups.com
To add to Luke's suggestion: from team RabbitMQ's experience on this list and elsewhere, core files
are significantly less useful than they sound. Monitoring [1], monitoring of applications and collecting log files
are more important and can help you troubleshoot and reason about all kinds of incidents.

Core files are occasionally useful and you should make it easy to locate them but 98 times out of 100 you
won't need them at all.


--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Arpan Gupta

unread,
Feb 26, 2018, 1:39:02 AM2/26/18
to rabbitmq-users
Does RabbitMQ generate core files automatically?

Thanks and Regards 
Arpan Gupta


On Friday, February 23, 2018 at 10:06:24 PM UTC+5:30, Michael Klishin wrote:
To add to Luke's suggestion: from team RabbitMQ's experience on this list and elsewhere, core files
are significantly less useful than they sound. Monitoring [1], monitoring of applications and collecting log files
are more important and can help you troubleshoot and reason about all kinds of incidents.

Core files are occasionally useful and you should make it easy to locate them but 98 times out of 100 you
won't need them at all.

On Fri, Feb 23, 2018 at 6:40 PM, Luke Bakken <lba...@pivotal.io> wrote:
Hi Arpan,

If you're talking about a core file, you should do some research into what other people do with docker containers. A quick search turned this up, for instance - https://github.com/moby/moby/issues/11740

There are articles online with information to do when you have an Erlang VM core file - https://www.erlang-solutions.com/blog/how-to-analyse-a-beam-core-dump.html

Finally, the RabbitMQ server log files should be the first place you check if you run into issues.

Thanks,
Luke
--
Staff Software Engineer
Pivotal / RabbitMQ

On Thursday, February 22, 2018 at 9:23:42 PM UTC-8, Arpan Gupta wrote:


s it possible to get Thread Dump and Heap Dump in case of a RabbitMQ Server?

To be Specific:

The RabbitMQ server I am referring to runs on a docker and I want to get Thread Dump and Heap Dump in such a case which would be useful in disaster analysis or other preventive alerts.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Arpan Gupta

unread,
Feb 26, 2018, 7:26:49 AM2/26/18
to rabbitmq-users
And Is there a way to produce Core dump files at runtime?

Thanks
Arpan Gupta

Luke Bakken

unread,
Feb 26, 2018, 11:06:53 AM2/26/18
to rabbitmq-users
Hi Arpan,

I'm assuming that you saw my and Michael's response with regard to Erlang VM core files. Just to re-iterate, they will probably be of little use in case an issue arises, and I want to stress the importance of monitoring your RabbitMQ implementation so that you can be aware of issues before they become serious.

> Does RabbitMQ generate core files automatically?
> And Is there a way to produce Core dump files at runtime?

I am assuming that you are running a Linux-based system. In addition to the reference I linked to earlier in this message thread, this could be useful:


It is not RabbitMQ's responsibility to configure your system to generate core files.

You may or may not be aware that the Erlang VM has a crash dump feature that is different from a core file - http://erlang.org/doc/apps/erts/crash_dump.html

Again, this will be of limited usefulness in the case of a serious event (though crash dumps are more useful than core files)

Thanks,
Luke

Arpan Gupta

unread,
Feb 26, 2018, 11:22:25 PM2/26/18
to rabbitmq-users
Thanks Luke, This was really helpful.

Arpan Gupta

unread,
Feb 27, 2018, 12:50:36 AM2/27/18
to rabbitmq-users
Is there a way to enable GC logging in RabbitMQ Servers?

Thanks 
Arpan Gupta

On Friday, February 23, 2018 at 10:53:42 AM UTC+5:30, Arpan Gupta wrote:

Arpan Gupta

unread,
Feb 27, 2018, 1:23:00 AM2/27/18
to rabbitmq-users
I know it is something Java specific but is there a way to get Garabage collector logs in case of RabbitMQ Server and if they would be of any use?

Michael Klishin

unread,
Feb 27, 2018, 7:22:53 AM2/27/18
to rabbitm...@googlegroups.com
There are GC rate metrics in the management UI/HTTP API. You cannot enable "GC logging" and I would
not do that anyway since GC in Erlang is not global (stop-the-world) and the rate can be high enough
to overwhelm the logger.

Again, from years of experience on this list what you want first and foremost is monitoring (including metrics such as queue lengths
and memory breakdown), log collection and monitoring
of your apps (including their runtime settings such as channel prefetch/QoS and publishing mode: transient vs. persistent).

Core dumps and GC activity are  rarely needed. Don't sweat it.

Scheduler context switching (also exposed in the management UI on the node page) and disk I/O are more important to monitor than GC activity
if you ask me.



--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages