Unable to start Rabbitmq on CentOS 7 (SELinux is enabled and is enforcing)

[Allan]

I have been trying to start RabbitMQ 3.5.6-1 on CentOS 7.
I have installed RabbitMQ 3.5.6-1 (from rabbitmq-server-3.5.6-1.noarch.rpm) on CentOS 7.
RabbitMQ fails to start showing errors shown (briefly) at the end of this email.
A look into SELinux log file shows "avc denied" (errors below).

How can I remedy this issue.



#grep -e denied /var/log/audit/audit.log

type=AVC msg=audit(1446475116.096:6602): avc:  denied  { read } for  pid=30524 comm="beam.smp" name=".erlang.cookie" dev="zfs" ino=63754 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
type=AVC msg=audit(1446475116.096:6603): avc:  denied  { read } for  pid=30525 comm="beam.smp" name=".erlang.cookie" dev="zfs" ino=63754 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
type=AVC msg=audit(1446475116.611:6604): avc:  denied  { add_name } for  pid=30579 comm="beam.smp" name="erl_crash.dump" scontext=system_u:system_r:rabbitmq_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=dir
type=AVC msg=audit(1446475577.682:6706): avc:  denied  { read } for  pid=32660 comm="beam.smp" name=".erlang.cookie" dev="zfs" ino=3173010 scontext


#sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28



[Errors on RabbitMQ start are shown below]
etrieving http://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm
Preparing...                          ################################# [100%]
    package erlang-solutions-1.0-1.noarch is already installed
Preparing...                          ################################# [100%]
Updating / installing...
   1:rabbitmq-server-3.5.6-1          ################################# [100%]
Starting rabbitmq-server (via systemctl):  Job for rabbitmq-server.service failed. See 'systemctl status rabbitmq-server.service' and 'journalctl -xn' for details.
                                                           [FAILED]
Mon Nov  2 17:46:19 AST 2015
Plugin configuration unchanged.

Applying plugin configuration to rabbit@somecomputer... failed.
 * Could not contact node rabbit@somecomputer.
   Changes will take effect at broker restart.
 * Options: --online  - fail if broker cannot be contacted.
            --offline - do not try to contact broker.
Mon Nov  2 17:46:19 AST 2015
Error: unable to connect to node rabbit@somecomputer: nodedown

DIAGNOSTICS
===========

attempted to contact: [rabbit@somecomputer]

rabbit@somecomputer:
  * connected to epmd (port 4369) on somecomputer
  * epmd reports: node 'rabbit' not running at all
                  no other nodes on somecomputer
  * suggestion: start the node

current node details:
- node name: 'rabbitmq-cli-33119@somecomputer'
- home dir: /var/lib/rabbitmq
- cookie hash: i6io7eruUKNmXyHsdfbNAQ==

....


Regards,
Allan.

[Jean-Sébastien Pédron]

On 02.11.2015 16:18, Allan wrote:
> #grep -e denied /var/log/audit/audit.log
>
> type=AVC msg=audit(1446475116.096:6602): avc: denied { read } for
> pid=30524 comm="beam.smp" name=".erlang.cookie" dev="zfs" ino=63754
> scontext=system_u:system_r:rabbitmq_t:s0
> tcontext=system_u:object_r:unlabeled_t:s0 tclass=file

Hi!

I am no SELinux expert but after some research, the error above means
that the application (the Erlang VM running RabbitMQ) is not allowed to
read .erlang.cookie. As I understand, this is unrelated to Unix
filesystem permissions.

So on startup, Erlang wants to read .erlang.cookie in the user's home
directory. With our RPM package, this is:
/var/lib/rabbitmq/.erlang.cookie

This file must be readable only by the owner, not the group or others.
But I suppose you need to configure SELinux to allow beam.smp to read
this file.

> type=AVC msg=audit(1446475116.611:6604): avc: denied { add_name } for
> pid=30579 comm="beam.smp" name="erl_crash.dump"
> scontext=system_u:system_r:rabbitmq_t:s0
> tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=dir

I believe you can ignore this access to erl_crash.dump. Erlang probably
tries to write this fail because it crashes after the failure to read
.erlang.cookie.

--
Jean-Sébastien Pédron
Pivotal / RabbitMQ

[Allan]

Thank you Jean-Sébastien for your input.

I am able to list the user as user rabbitmq using the command below.
$ sudo -u rabbitmq ls -ltr /var/lib/rabbitmq/.erlang.cookie
-r--------. 1 rabbitmq rabbitmq 20 Nov  2 00:00 /var/lib/rabbitmq/.erlang.cookie

I can also successfully view the contents of this file using the command below
$ sudo -u rabbitmq less /var/lib/rabbitmq/.erlang.cookie


Allan.
 

[Michael Klishin]

"Applying plugin configuration" includes extracting plugin archives into code paths on disk.

So that's a fairly obvious thing that RabbitMQ process should be allowed to do by SELinux.

The audit log suggests that the process tries to read .erlang.cookie and gets denied, too. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ