Cannot install rabbitmq-server on ubuntu server 16.04.4

1,514 views
Skip to first unread message

Igor Komar

unread,
May 30, 2018, 7:37:39 AM5/30/18
to rabbitmq-users

Hi, I'm trying to install rabbitmq-server on ubuntu Server 16.04 as:

apt-get install rabbitmq-server -y

during installation I'm getting error:


May 30 06:27:26 v systemd[1]: Starting RabbitMQ Messaging Server...
May 30 06:27:27 v rabbitmq[31277]: Waiting for rabbit@v ...
May 30 06:27:27 v rabbitmq[31277]: pid is 31285 ...
May 30 06:27:28 v systemd[1]: rabbitmq-server.service: Main process exited, code=exited, status=140/n/a
May 30 06:27:29 v rabbitmq[31277]: Error: process_not_running
May 30 06:27:29 v systemd[1]: rabbitmq-server.service: Control process exited, code=exited status=2
May 30 06:27:29 v systemd[1]: Failed to start RabbitMQ Messaging Server.
May 30 06:27:29 v systemd[1]: rabbitmq-server.service: Unit entered failed state.
May 30 06:27:29 v systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.
dpkg: error processing package rabbitmq-server (--configure):
 subprocess installed post-installation script returned error exit status 
sudo rabbitmqctl status saying:
Status of node rabbit@v ...
Error: unable to connect to node rabbit@v: nodedown

DIAGNOSTICS
===========

attempted to contact: [rabbit@v]

rabbit@v:
  * connected to epmd (port 4369) on v
  * epmd reports: node 'rabbit' not running at all
                  no other nodes on v
  * suggestion: start the node

current node details:
- node name: 'rabbitmq-cli-32749@v'
- home dir: /var/lib/rabbitmq
- cookie hash: KHRxM68AKD22OGcXr68lyQ==

It's a real server for testing our application. If I'm installing rabbitmq in virtualbox also on ubuntu server 16.04 everything ok. 
Can anybody help with it?

Michael Klishin

unread,
May 30, 2018, 7:40:40 AM5/30/18
to rabbitm...@googlegroups.com
See server logs [1] and redirects of stdout and stderr that the Debian package stores in the same directory (IIRC).


--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Michael Klishin

unread,
May 30, 2018, 7:41:51 AM5/30/18
to rabbitm...@googlegroups.com
Also no details were provided about what RabbitMQ version is used, what repository it is provisioned from,
what Erlang versions is used and so on. Please help others help you. We don't guess on this list.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ
Message has been deleted

Igor Komar

unread,
May 30, 2018, 7:54:37 AM5/30/18
to rabbitmq-users
thank you for your reply, I installed rabbitmq as sudo apt-get install rabbitmq-server, 
it writing version: "RabbitMQ 3.5.7", 
erlang looks like 18.3:
$ cat /usr/lib/erlang/releases/18/OTP_VERSION 
showing 18.3

$ rabbitmqctl environment:
Crash dump is being written to: erl_crash.dump...done


среда, 30 мая 2018 г., 14:41:51 UTC+3 пользователь Michael Klishin написал:
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Michael Klishin

unread,
May 30, 2018, 7:58:23 AM5/30/18
to rabbitm...@googlegroups.com
What is in server logs? Is there *anything* else that CLI tools output? If the erl_crash.dump is the only
clue there is (it is almost never the case), can at the very least it be uploaded?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Michael Klishin

unread,
May 30, 2018, 8:00:44 AM5/30/18
to Igor Komar, rabbitm...@googlegroups.com
I do not provide any kind of 1-on-1 support. Please CC the list.

> bash: fork: Cannot allocate memory

is pretty unambiguous: the runtime cannot start (fork) a subprocess.

How much memory does the VM in question have available and free? (as in `free -m`)

On Wed, May 30, 2018 at 2:52 PM, Igor Komar <mymt...@gmail.com> wrote:
thank you for your reply, I installed rabbitmq as sudo apt-get install rabbitmq-server, 
it writing version: "RabbitMQ 3.5.7", 
erlang looks like 18.3:
cat /usr/lib/erlang/releases/18/OTP_VERSION 
showing: 18.3

rabbitmqctl environment:
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory

when run again
rabbitmqctl environment:
 Crash dump is being written to: erl_crash.dump...

среда, 30 мая 2018 г., 14:40:40 UTC+3 пользователь Michael Klishin написал:
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
MK

Staff Software Engineer, Pivotal/RabbitMQ

Igor Komar

unread,
May 30, 2018, 8:10:09 AM5/30/18
to rabbitmq-users
free -m
              total        used        free      shared  buff/cache   available
Mem:           1024         215         242           7         565         677
Swap:             0           0           0


среда, 30 мая 2018 г., 15:00:44 UTC+3 пользователь Michael Klishin написал:

Igor Komar

unread,
May 30, 2018, 8:21:00 AM5/30/18
to rabbitmq-users
log files are empty,
root@v:~# cat /var/log/rabbitmq/startup_err 
root@v:~# cat /var/log/rabbitmq/startup_log 

I even don't know how to find other logs, rabbitmqctl environment hangs my terminal and shows Cannot allocate memory



среда, 30 мая 2018 г., 15:00:44 UTC+3 пользователь Michael Klishin написал:

Michael Klishin

unread,
May 30, 2018, 8:30:03 AM5/30/18
to rabbitm...@googlegroups.com
Log file location is documented [1]. The kernel has a number of log files used by different tools and subsystems under /var/log. There are tools [4] that will collect all of them into a single tarball.

If there are no logs then the node didn't have a chance to start at all
or failed very early on. The fork failure message from the shell is definitely suspicious.

1G is plenty of RAM for a RabbitMQ node but there can be other system limits and tools
such as SELinux that can prevent a process from doing much.

3.5.x is out of any kind of support, so in order to continue getting
help on this list please move to installing a 3.7.x version [2] with a compatible Erlang/OTP one [3].

epmd is not the Erlang runtime, it's a port mapping daemon used by Erlang nodes
and CLI tools for discovery. Erlang VM processes are named "beam".

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Igor Komar

unread,
May 30, 2018, 8:44:38 AM5/30/18
to rabbitmq-users
I already tried to install rabbitmq as described here http://www.rabbitmq.com/install-debian.html but installation ended with same error
$RABBITMQ_LOGS variable is empty
I saw tickets where problem was in hostname or hosts.

maybe will help:
root@v:~# cat /etc/hosts ff02::1 ip6-allnodes ff02::2 ip6-allrouters 127.0.0.1 localhost.localdomain localhost # Auto-generated hostname. Please do not remove this comment. 80.87.197.113 v.someusersurname.fvds.ru v ::1 localhost ip6-localhost ip6-loopback

and

root@v:~# cat /etc/hostname v

just found that next 2 log files not empty, as earlier
$ cat /var/log/rabbitmq/rabbit\@v-sasl.log 

=CRASH REPORT==== 30-May-2018::07:46:11 ===
  crasher:
    initial call: rabbit_disk_monitor:init/1
    pid: <0.79.0>
    registered_name: rabbit_disk_monitor
    exception exit: {enomem,[{erlang,open_port,
                                     [{spawn,"/bin/sh -s unix:cmd 2>&1"},
                                      [stream]],
                                     []},
                             {os,start_port_srv_handle,1,
                                 [{file,"os.erl"},{line,313}]},
                             {os,start_port_srv_loop,0,
                                 [{file,"os.erl"},{line,329}]}]}
      in function  gen_server:terminate/7 (gen_server.erl, line 826)
    ancestors: [rabbit_disk_monitor_sup,rabbit_sup,<0.71.0>]
    messages: []
    links: [<0.78.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 2586
    stack_size: 27
    reductions: 9000
  neighbours:

=SUPERVISOR REPORT==== 30-May-2018::07:46:11 ===
     Supervisor: {local,rabbit_disk_monitor_sup}
     Context:    child_terminated
     Reason:     {enomem,[{erlang,open_port,
                                  [{spawn,"/bin/sh -s unix:cmd 2>&1"},
                                   [stream]],
                                  []},
                          {os,start_port_srv_handle,1,
                              [{file,"os.erl"},{line,313}]},
                          {os,start_port_srv_loop,0,
                              [{file,"os.erl"},{line,329}]}]}
     Offender:   [{pid,<0.79.0>},
                  {name,rabbit_disk_monitor},
                  {mfargs,{rabbit_disk_monitor,start_link,[50000000]}},
                  {restart_type,{transient,1}},
                  {shutdown,4294967295},
                  {child_type,worker}]



root@v:~# cat /var/log/rabbitmq/rabbit\@v.log 

=INFO REPORT==== 30-May-2018::07:44:50 ===
Starting RabbitMQ 3.5.7 on Erlang 18.3
Copyright (C) 2007-2015 Pivotal Software, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

=INFO REPORT==== 30-May-2018::07:44:50 ===
node           : rabbit@v
home dir       : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config (not found)
cookie hash    : KHRxM68AKD22OGcXr68lyQ==
log            : /var/log/rabbitmq/rab...@v.log
sasl log       : /var/log/rabbitmq/rab...@v-sasl.log
database dir   : /var/lib/rabbitmq/mnesia/rabbit@v

=INFO REPORT==== 30-May-2018::07:44:51 ===
Memory limit set to 409MB of 1024MB total.

=INFO REPORT==== 30-May-2018::07:44:51 ===
Disk free limit set to 50MB

=INFO REPORT==== 30-May-2018::07:44:51 ===
Limiting to approx 924 file handles (829 sockets)

=INFO REPORT==== 30-May-2018::07:44:51 ===
FHC read buffering:  ON
FHC write buffering: ON

=INFO REPORT==== 30-May-2018::07:44:51 ===
Database directory at /var/lib/rabbitmq/mnesia/rabbit@v is empty. Initialising from scratch... 

=INFO REPORT==== 30-May-2018::07:44:52 ===
Priority queues enabled, real BQ is rabbit_variable_queue

=INFO REPORT==== 30-May-2018::07:44:52 ===
Adding vhost '/'

=INFO REPORT==== 30-May-2018::07:44:52 ===
Creating user 'guest'

=INFO REPORT==== 30-May-2018::07:44:52 ===
Setting user tags for user 'guest' to [administrator]

=INFO REPORT==== 30-May-2018::07:44:52 ===
Setting permissions for 'guest' in '/' to '.*', '.*', '.*'

=INFO REPORT==== 30-May-2018::07:44:52 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 30-May-2018::07:44:52 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index

=WARNING REPORT==== 30-May-2018::07:44:52 ===
msg_store_persistent: rebuilding indices from scratch

=INFO REPORT==== 30-May-2018::07:44:52 ===
started TCP Listener on [::]:5672

=INFO REPORT==== 30-May-2018::07:44:52 ===
Server startup complete; 0 plugins started.

=ERROR REPORT==== 30-May-2018::07:46:11 ===
** Generic server rabbit_disk_monitor terminating 
** Last message in was update
** When Server state == {state,"/var/lib/rabbitmq/mnesia/rabbit@v",50000000,
                               31302238208,100,10000,#Ref<0.0.1.2353>,false,
                               true}
** Reason for termination == 
** {{enomem,[{erlang,open_port,
                     [{spawn,"/bin/sh -s unix:cmd 2>&1"},[stream]],
                     []},
             {os,start_port_srv_handle,1,[{file,"os.erl"},{line,313}]},
             {os,start_port_srv_loop,0,[{file,"os.erl"},{line,329}]}]},
    [{os,unix_cmd,1,[{file,"os.erl"},{line,249}]},
     {os,cmd,1,[{file,"os.erl"},{line,217}]},
     {rabbit_disk_monitor,get_disk_free,2,
                          [{file,"src/rabbit_disk_monitor.erl"},{line,194}]},
     {rabbit_disk_monitor,internal_update,1,
                          [{file,"src/rabbit_disk_monitor.erl"},{line,173}]},
     {rabbit_disk_monitor,handle_info,2,
                          [{file,"src/rabbit_disk_monitor.erl"},{line,145}]},
     {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,615}]},
     {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,681}]},
     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}

=INFO REPORT==== 30-May-2018::07:46:11 ===
Disabling disk free space monitoring on unsupported platform:
{{'EXIT',{enomem,[{erlang,open_port,
                          [{spawn,"/bin/sh -s unix:cmd 2>&1"},[stream]],
                          []},
                  {os,start_port_srv_handle,1,[{file,"os.erl"},{line,313}]},
                  {os,start_port_srv_loop,0,[{file,"os.erl"},{line,329}]}]}},
 1073741824}


среда, 30 мая 2018 г., 15:30:03 UTC+3 пользователь Michael Klishin написал:

Michael Klishin

unread,
May 30, 2018, 8:55:56 AM5/30/18
to rabbitm...@googlegroups.com
RabbitMQ tries to run `/bin/sh -s unix:cmd 2>&1` to monitor disk space and
that fails with `enomem`, which confirms the fork failure previously reported by CLI tools.

Apparently in 3.5.7 disk space monitor failure is fatal. It's not in late 3.6.x and 3.7.x versions.

sasl log       : /var/log/rabbitmq/rabbit@v-sasl.log
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Igor Komar

unread,
May 30, 2018, 9:06:16 AM5/30/18
to rabbitmq-users
is this error happen when hard disk have no space? 
If I correctly got your answer, if I will install rabbitmq 3.7 e.g. It will work? I already tried earlier to reinstall rabbitmq, and install it from as shown here https://www.rabbitmq.com/install-debian.html#bintray-apt-repo - but saw errors also. 

среда, 30 мая 2018 г., 15:55:56 UTC+3 пользователь Michael Klishin написал:

Igor Komar

unread,
May 30, 2018, 9:08:07 AM5/30/18
to rabbitmq-users
30G available
root@v:~# df -h --total
Filesystem      Size  Used Avail Use% Mounted on
/dev/simfs       30G  869M   30G   3% /
devtmpfs        512M     0  512M   0% /dev
tmpfs           512M     0  512M   0% /dev/shm
tmpfs           512M  7.6M  505M   2% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           512M     0  512M   0% /sys/fs/cgroup
none            512M     0  512M   0% /run/shm
total            33G  877M   32G   3% -


среда, 30 мая 2018 г., 15:55:56 UTC+3 пользователь Michael Klishin написал:

Michael Klishin

unread,
May 30, 2018, 9:21:22 AM5/30/18
to rabbitm...@googlegroups.com
I did not say that. What I said was that RabbitMQ’s free disk monitor tries to *start a subprocess* and it fails with the error of ENOMEM.

The node detects it has about 1 GB of memory available according to server logs. Something prevents subprocesses from being started (ENOMEM is likely used for more than virtual memory). Perhaps kernel log files will provide some clues.

I don’t know if RabbitMQ 3.7.5 will work. Clearly there’s something unusual about your VM. I do know that 3.5.x (in fact, technically 3.6.x) is out of support and the only piece of advice we have for it is: please move to 3.7 first.

Igor Komar

unread,
May 30, 2018, 9:36:44 AM5/30/18
to rabbitmq-users
ok, thank you

среда, 30 мая 2018 г., 16:21:22 UTC+3 пользователь Michael Klishin написал:
log            : /var/log/rab...@v.log

Gabriel-Iulian Dumbrava

unread,
Aug 8, 2018, 11:22:03 AM8/8/18
to rabbitmq-users
I had a same issue, but on 18.04 and I managed to get over it by upgrading rabbit to version 3.7 which is compatible with the latest erlang version 21.x.

I followed the instructions provided here.

I hope this works for you! Took me a few hours, as they were no errors in rabbit logs, and the erlang.dump was completely not useful.

Michael Klishin

unread,
Aug 8, 2018, 2:46:38 PM8/8/18
to rabbitm...@googlegroups.com
That's the guide that contains installation instructions on Debian and Debian derivatives.

Erlang crash dumps are meant to provide access to very low level details about the state of the node.
Very often you don't get there since the problem is with something else.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Amani Masila

unread,
Nov 22, 2018, 2:57:52 AM11/22/18
to rabbitmq-users
Hello, 

please I need help am getting an error which i do not understanf

After this operation, 0 B of additional disk space will be used.
Setting up rabbitmq-server (3.7.9-1) ...
Job for rabbitmq-server.service failed because the control process exited with error code.
See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.
invoke-rc.d: initscript rabbitmq-server, action "start" failed.
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Thu 2018-11-22 10:55:11 EAT; 8ms ago
  Process: 24268 ExecStart=/usr/lib/rabbitmq/bin/rabbitmq-server (code=exited, status=1/FAILURE)
 Main PID: 24268 (code=exited, status=1/FAILURE)
dpkg: error processing package rabbitmq-server (--configure):
 installed rabbitmq-server package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 rabbitmq-server
E: Sub-process /usr/bin/dpkg returned an error code (1)

Michael Klishin

unread,
Nov 22, 2018, 7:32:20 PM11/22/18
to rabbitm...@googlegroups.com
This list uses one question per thread. Please see and post server logs, stdout and stderr files produced
by the Debian package as well as any relevant OS logs or we won't bee able to help you.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages