RabbitMQ Server Crash

1,030 views
Skip to first unread message

Dileep

unread,
Apr 2, 2019, 2:28:09 AM4/2/19
to rabbitmq-users
Hello All,

I am trying to run a standalone rabbitmq server on xilinx petalinux platform (i.e. armhf). and the rabbitmq server is crashing.
I am not able to figure out if the crash is being caused by Free disk space monitor or anything else.

RabbitMq version: RabbitMQ 3.7.0
Erlang version : 20.1.7

Both the pre-built versions were taken.

The things which i have already tried:
1) Deleting the contents inside this folder "/home/root/rabbitmq_server-3.7.0/var/lib/rabbitmq/mnesia"
    and restarting the server.
2) Tried launching the server using different rabbitmq servers i.e. 3.7.5 & 3.7.6


Below are the logs:

2019-03-28 12:52:13.962 [info] <0.33.0> Application lager started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:13.986 [info] <0.5.0> Log file opened with Lager
2019-03-28 12:52:15.401 [info] <0.33.0> Application xmerl started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:15.770 [info] <0.33.0> Application inets started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:15.771 [info] <0.33.0> Application crypto started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:15.772 [info] <0.33.0> Application recon started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:15.773 [info] <0.33.0> Application jsx started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.381 [info] <0.33.0> Application mnesia started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.410 [info] <0.33.0> Application os_mon started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.412 [info] <0.33.0> Application asn1 started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.413 [info] <0.33.0> Application public_key started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.725 [info] <0.33.0> Application ssl started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.750 [info] <0.33.0> Application ranch started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.751 [info] <0.33.0> Application ranch_proxy_protocol started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.752 [info] <0.33.0> Application rabbit_common started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:16.802 [info] <0.182.0>
 Starting RabbitMQ 3.7.0 on Erlang 20.1.7
 Copyright (C) 2007-2017 Pivotal Software, Inc.
 Licensed under the MPL.  See http://www.rabbitmq.com/
2019-03-28 12:52:16.921 [info] <0.182.0>
 node           : rabbit@mz7010-fmccc-2018-2
 home dir       : /home/root
 config file(s) : (none)
 cookie hash    : Dv0zb7NodaPP2fCzya/h+g==
 log(s)         : /home/root/rabbitmq_server-3.7.0/var/log/rabbitmq/rab...@mz7010-fmccc-2018-2.log
                : /home/root/rabbitmq_server-3.7.0/var/log/rabbitmq/rabbit@mz7010-fmccc-2018-2_upgrade.log
 database dir   : /home/root/rabbitmq_server-3.7.0/var/lib/rabbitmq/mnesia/rabbit@mz7010-fmccc-2018-2
2019-03-28 12:52:27.706 [error] <0.190.0> CRASH REPORT Process vm_memory_monitor with 0 neighbours exited with reason: bad return value: {command_not_found,"getconf"} in gen_server:init_it/6 line 357
2019-03-28 12:52:27.708 [error] <0.189.0> Supervisor vm_memory_monitor_sup had child vm_memory_monitor started with vm_memory_monitor:start_link(0.4, #Fun<rabbit_alarm.0.123076882>, #Fun<rabbit_alarm.1.12
2019-03-28 12:52:27.736 [info] <0.192.0> FORMAT ERROR: "Free disk space monitor encountered an error (e.g. failed to parse output from OS tools): ~p, retries left: ~s~n" [{{'EXIT',{badarg,[{erlang,list_to
2019-03-28 12:52:27.738 [info] <0.195.0> Limiting to approx 924 file handles (829 sockets)
2019-03-28 12:52:27.739 [info] <0.196.0> FHC read buffering:  OFF
2019-03-28 12:52:27.739 [info] <0.196.0> FHC write buffering: ON
2019-03-28 12:52:27.743 [info] <0.182.0> Node database directory at /home/root/rabbitmq_server-3.7.0/var/lib/rabbitmq/mnesia/rabbit@mz7010-fmccc-2018-2 is empty. Assuming we need to join an existing clust
2019-03-28 12:52:27.744 [info] <0.182.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config
2019-03-28 12:52:27.744 [info] <0.182.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config
2019-03-28 12:52:27.745 [info] <0.182.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping randomized startup delay.
2019-03-28 12:52:27.745 [info] <0.182.0> All discovered existing cluster peers:
2019-03-28 12:52:27.745 [info] <0.182.0> Discovered no peer nodes to cluster with
2019-03-28 12:52:27.769 [info] <0.33.0> Application mnesia exited with reason: stopped
2019-03-28 12:52:27.835 [info] <0.33.0> Application mnesia started on node 'rabbit@mz7010-fmccc-2018-2'
2019-03-28 12:52:28.033 [info] <0.182.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2019-03-28 12:52:28.125 [info] <0.182.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2019-03-28 12:52:28.217 [info] <0.182.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2019-03-28 12:52:28.218 [info] <0.182.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2019-03-28 12:52:28.224 [info] <0.182.0> Priority queues enabled, real BQ is rabbit_variable_queue
2019-03-28 12:52:28.245 [info] <0.364.0> Starting rabbit_node_monitor
2019-03-28 12:52:28.255 [warning] <0.383.0> Memory monitor process not yet started: {gen_server,call,[vm_memory_monitor,get_cached_process_memory_and_limit,infinity]}
2019-03-28 12:52:28.258 [error] <0.383.0> CRASH REPORT Process rabbit_memory_monitor with 0 neighbours exited with reason: bad return value: {command_not_found,"getconf"} in gen_server2:init_it/6 line 593
2019-03-28 12:52:28.260 [error] <0.382.0> Supervisor rabbit_memory_monitor_sup had child rabbit_memory_monitor started with rabbit_memory_monitor:start_link() at undefined exit with reason bad return valu
2019-03-28 12:52:28.263 [error] <0.181.0> CRASH REPORT Process <0.181.0> with 0 neighbours exited with reason: {error,{{shutdown,{failed_to_start_child,rabbit_memory_monitor,{bad_return_value,{command_not
2019-03-28 12:52:28.266 [info] <0.33.0> Application rabbit exited with reason: {error,{{shutdown,{failed_to_start_child,rabbit_memory_monitor,{bad_return_value,{command_not_found,"getconf"}}}},{child,unde

Thanks in advance

Regards,
Dileep


Grigory Starinkin

unread,
Apr 2, 2019, 3:09:07 AM4/2/19
to rabbitm...@googlegroups.com
Hi Dileep,

It appears to be that platform on which you are trying to start RabbitMQ doesn’t have “getconf” command (or PATH env variable is not configured properly).

getconf is needed to get OS page size to set memory limit.

Kind regards,
Grigory Starinkin 
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Luke Bakken

unread,
Apr 2, 2019, 10:55:29 AM4/2/19
to rabbitmq-users
Thanks for answering Grigory.

Another workaround is to create a "dummy" getconf command that returns 4096:

#!/bin/sh
echo 4096

Save that as /bin/getconf and ensure it is executable.

I will make RabbitMQ more forgiving if getconf is not available (which is extremely rare).

Thanks,
Luke
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

Dileep

unread,
Apr 3, 2019, 4:36:17 AM4/3/19
to rabbitmq-users
Thanks for answering Grigory & Luke. "getconf" wasn't enabled in the platform. After doing quick workaround as suggested by Luke, the rabbitmq server is running without a crash.


On Tuesday, 2 April 2019 12:39:07 UTC+5:30, Grigory Starinkin wrote:
Hi Dileep,

It appears to be that platform on which you are trying to start RabbitMQ doesn’t have “getconf” command (or PATH env variable is not configured properly).

getconf is needed to get OS page size to set memory limit.

Kind regards,
Grigory Starinkin 
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages