Unable to configure Rabbitmq with automatic rabbitmq log rotation and log archiving

[badrira...@gmail.com]

Hi team, 

I am using RabbitMQ 3.6.10, Erlang 19.3.6.4 and I am trying to configure the automatic rabbitmq log rotation and log archiving based on size (For Ex: rotate the rabbitmq log file if the size of the file reaches 1Gb and archive the old log files). I checked online for rabbitmq.config changes to achieve this but unable to find the same. 

Can you kindly let me know the configuration changes needed in the rabbitmq.config file to achieve log rotation and log archiving?

[Michael Klishin]

Please help others help you by providing more relevant information.

What specifically does your config file look like? How was the test conducted?

RabbitMQ 3.6.x has been out of support since May 2018. 3.7.x makes it fairly easy to configure log rotation [1]

and RabbitMQ Debian package will set up logrotate at installation time.

Consider upgrading to 3.7.11 first [2][3].

1. https://www.rabbitmq.com/logging.html#log-rotation

2. https://www.rabbitmq.com/changelog.html

3. https://www.rabbitmq.com/upgrade.html

[badrira...@gmail.com]

Hi MK,

   

 I have installed RabbitMQ 3.6.10, Erlang 19.3.6.4 on Windows 10 system. I have configured few outgoing shovels to RabbitMQ broker on another system. Over a period of time if there are any shovels / network issue,  the RabbitMq log file size keeps growing indefinitely

 I am unable to view the log file in any text editor if the log file size crosses 1 GB. Also if unnoticed, the log file size eats up the disk space in the worst case. Hence i wanted to rotate the RabbitMq log file periodically, so that file size doesn't cross 1 GB or eats up the system disk space.

There were few solutions online for higher versions of RabbitMq but i couldn't find the rabbitmq.config configuration for RabbitMQ 3.6.10.

  Kindly let me know if there is any log rotation and archiving configuration for RabbitMQ 3.6.10

[Michael Klishin]

Logrotate or similar external rotation tools are the answer for 3.6.x on Windows users.

E.g. [1] looks reasonably recently updated.

1. https://github.com/plecos/logrotatewin

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[MH]

I'm having troubles with logs rotating also. with the %D0 for rotate setting in the advanced.config below it rotates nightly but nothing new gets written to the new log file, old one is renamed to complete.log.0 which after a fresh start it does log things, it just doesnt put anything in the log after a log rotate. I get the same with a manual rabbitmqctl rotate_logs command, nothing new in new file.

rabbit 3.7.11 erlang 21.2

windows server 2019

advanced.config below with nothing in rabbitmq.config:

[{rabbit, [

        {log, [

            {file, [{file, "D:\\RabbitMQ\\log\\complete.log"}, %% log.file

                    {level, debug}       %% log.file.info

                    ]}

        ]}

    ]}].

windows service was removed and recreated with the below vars run in the same terminal before the service was created

set RABBITMQ_BASE=D:\RabbitMQ

set RABBITMQ_CONFIG_FILE=D:\RabbitMQ\config\rabbitmq

set RABBITMQ_ADVANCED_CONFIG_FILE=D:\RabbitMQ\config\advanced

set RABBITMQ_MNESIA_BASE=D:\RabbitMQ\db

set RABBITMQ_LOG_BASE=D:\RabbitMQ\log

[MH]

I also get the same behavior when using default everything and nothing specific in the config files where the logs and db goes into the appdata folder, after a rotate_logs nothing gets written to the new or old log file. Last line in the rotated log is 

2019-02-21 11:21:34.613 [info] <0.1261.0> Log file rotation forced

[Michael Klishin]

We will investigate. We have seen a few reports like this recently so chances are it is not something environment-specific.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luke Bakken]

Hello,

I have been able to reproduce these issues with log rotation on Windows and I'm narrowing in on the cause.

Thanks,

Luke

[Michael Klishin]

This was a problem in the logging library we use, Luke addressed it [1]. RabbitMQ will ship an updated version

starting with 3.7.14.

1. https://github.com/erlang-lager/lager/pull/493

[badrira...@gmail.com]

Hi Michael,

    I have configured the log rotate settings in Advanced.config file as below in RabbitMQ 3.7.12  on Erlang 21.3 on Windows machine,

      

     {lager, [

             {handlers, [

                         {lager_file_backend, [{file, "MyRabbit.log"},

                                   { level, debug},

                                      {date, "$H00"},

                                     {size, 10485760},

                                    {count, 3}

                        ]}]}

    ]}

 

However,  log files getting rotated for the first hour but from the next hour there is no log collection happening and created  new file remains empty. 

for example as shown in the below screen , log.0 and log.1 are new files of 2 hours but both are empty.

So, As you said that a problem in the logging library we use and RabbitMQ will ship an updated version starting with 3.7.14

 Is it expected behavior in 3.7.12 ? 

Can I know about 3.7.14 version release plan (Approximately) ? 

Thanks 

  Badri

[Luke Bakken]

Hi Badri,

RabbitMQ 3.7.14 has been available for almost a month - 

https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.7.14

Thanks -
Luke

[badri ramesh]

Hi Luke,

    I am using Rabbitmq v3.7.16 with Erlang 21.3 on windows machine,

I have configured Log levels as below in my advanced.config file 

         {log_levels, [{connection, debug}, {channel, debug}]} and

   

I have configured the automatic rabbitmq log rotation and log archiving based on size (1 GB) as below,

     {lager, [

             {handlers, [

                         {lager_file_backend, [{file, "AutomaticRabbitmq.log"},

                                   {level, debug},

                                   {date, ""},

                                   {size, 1073741824},

                                   {count, 6}

                        ]}]}

    ]}

and I observed that the log rotation is happening properly but only DEBUG logs are clearing from the original file and INFO, WARNING logs are not clearing. Please see the attached sample snippet for your reference.

Can you please tell me how can I achieve this ? 

Thanks.