Logging problem after failed logrotate on CentOS

[Jake Cee]

Hello,

I'm having problems getting logging/logrotate to work properly on CentOS 7.2.  I have Erlang erlang-18.3-1.el7.centos.x86_64 and RabbitMQ rabbitmq-server-3.6.2-1.noarch.rpm installed, clustered, and (apparently, so far) running fine on three servers.   But, when my first weekly logrotate ran, I got this error:

"""
/etc/cron.daily/logrotate:

su: avc.c:74: avc_context_to_sid_raw: Assertion `avc_running' failed.
/usr/sbin/rabbitmqctl: line 47: 23171 Aborted                 su rabbitmq -s /bin/sh -c "/usr/lib/rabbitmq/bin/${

SCRIPT} ${CMDLINE}"
error: error running shared postrotate script for '/var/log/rabbitmq/*.log '
"""

Examining my /var/log/rabbitmq directory, I see:

# ls -la
total 104
drwxrwxr-x. 2 rabbitmq rabbitmq  4096 Jul 10 03:49 .
...
-rw-r--r--. 1 rabbitmq rabbitmq     0 Jul 10 03:49 rab...@rabbit3.log
-rw-r--r--. 1 rabbitmq rabbitmq 69790 Jul 11 14:23 rab...@rabbit3.log-20160710
...

Which shows me that the rotation happened on the 10th at 3:25 (at the time of my above error), but that the server appears to still be writing to the rotated file (as evidenced by the July 11th time stamp).

My /etc/logroate.d/rabbitmq-server file looks like this:

$ cat /etc/logrotate.d/rabbitmq-server
/var/log/rabbitmq/*.log {
    su rabbitmq rabbitmq
        weekly
        missingok
        rotate 20
        compress
        delaycompress
        notifempty
        sharedscripts
        postrotate
            /usr/sbin/rabbitmqctl rotate_logs > /dev/null
        endscript
}

This is the standard that came from the RPM except that I had to add the top line due to the fact that I run SELINUX enforcing on all my systems and was getting this error without it:

# logrotate --force /etc/logrotate.d/rabbitmq-server
error: skipping "/var/log/rabbitmq/rab...@rabbit2.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.

With the config file as above, when I run logrotate --force now, it doesn't fail, but, again the server appears to be writing to the old file.  If I just manually run this, I also get no error but still no writing to the new file:

#  /usr/sbin/rabbitmqctl rotate_logs
Reopening logs for node rabbit@rabbit2 ...

Any help or advice would be highly appreciated. I don't want to deploy this new cluster into production without reliable logging.

Thanks in advance!
Jake

[Karl Nilsson]

Hi,

I know next to nothing about SELinux but found a link here that appears relevant or at least worth reading through: https://bugzilla.redhat.com/show_bug.cgi?id=1298523

Cheers

Karl

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Staff Software Engineer, Pivotal/RabbitMQ

[Jake Cee]

Thanks Karl. 

I had seen that bug report in my searching but after you also posted it, this time I read more closely and followed the trail to learn this: "It is going to be addressed in rhel-7.3.0."  Which lead me to believe it's a RHEL/SELinux policy problem.

Since I need to get this working before the RHEL/CentOS 7.3 release, I'm going to try to change the SELinux policy to allow this, via audit2allow.  If it works (I'll know tomorrow), I'll post the steps I used for others' benefit.

Cheers,
Jake

# logrotate --force /etc/logrotate.d/rabbitmq-server
error: skipping "/var/log/rabbitmq/rabbit@rabbit2.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.

With the config file as above, when I run logrotate --force now, it doesn't fail, but, again the server appears to be writing to the old file.  If I just manually run this, I also get no error but still no writing to the new file:

#  /usr/sbin/rabbitmqctl rotate_logs
Reopening logs for node rabbit@rabbit2 ...

Any help or advice would be highly appreciated. I don't want to deploy this new cluster into production without reliable logging.

Thanks in advance!
Jake

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.