RabbitMQ and syslog

[Giorgos Flourentzos]

Hello,

Apologies in advance if this is a well-known matter. I couldn't find anything in the documentation and forum.

I am looking to connect RabbitMQ with syslog (rsyslog preferably) so that the RabbitMQ server running on my Ubuntu machine can send well-formatted messages to syslog. Does RabbitMQ already send messages to syslog, is there any configuration or plug in that needs setting up?

Just to give you the full picture (as a use case), I am using Papertrail and have set up syslog to forward errors to Papertrail.

Many many thanks in advance!

Best regards,

George

[Simon MacMullen]

I'm afraid RabbitMQ doesn't connect to syslog directly.

It is (I think) possible to install the lager plugin:

http://www.rabbitmq.com/community-plugins.html
https://github.com/basho/lager

and then use lager's syslog plugin:

https://github.com/basho/lager_syslog

(which you'd need to compile to a RabbitMQ plugin yourself) to get
messages to syslog.

I don't know if anyone has done this. Does anyone want to share their
experiences?

Cheers, Simon

On 17/11/14 14:15, Giorgos Flourentzos wrote:
> Hello,
>
> Apologies in advance if this is a well-known matter. I couldn't find
> anything in the documentation and forum.
>
> I am looking to connect RabbitMQ with syslog (rsyslog preferably) so
> that the RabbitMQ server running on my Ubuntu machine can send
> well-formatted messages to syslog. Does RabbitMQ already send messages
> to syslog, is there any configuration or plug in that needs setting up?
> Just to give you the full picture (as a use case), I am using Papertrail

> <https://papertrailapp.com/> and have set up syslog to forward errors to

> Papertrail.
>
> Many many thanks in advance!
>
> Best regards,
> George
>

> --
> You received this message because you are subscribed to the Google
> Groups "rabbitmq-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to rabbitmq-user...@googlegroups.com
> <mailto:rabbitmq-user...@googlegroups.com>.
> To post to this group, send email to rabbitm...@googlegroups.com
> <mailto:rabbitm...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Peter Lemenkov]

2014-11-17 17:29 GMT+03:00 Simon MacMullen <si...@rabbitmq.com>:
> I'm afraid RabbitMQ doesn't connect to syslog directly.

Is it possible to log to stdout instead? If yes, then systemd can do
the trick - it captures everything from stdout/stderr and redirects it
wherever one wants (Journald, syslog, files, sockets).


--
With best regards, Peter Lemenkov.

[Simon MacMullen]

On 17/11/14 14:36, Peter Lemenkov wrote:
> 2014-11-17 17:29 GMT+03:00 Simon MacMullen <si...@rabbitmq.com>:
>> I'm afraid RabbitMQ doesn't connect to syslog directly.
>
> Is it possible to log to stdout instead? If yes, then systemd can do
> the trick - it captures everything from stdout/stderr and redirects it
> wherever one wants (Journald, syslog, files, sockets).

It is. Sort of.

You need to hack the rabbitmq-server startup script (which is at
/usr/lib/rabbitmq/lib/rabbitmq_server-X.Y.Z/bin on RPM):

diff -r 0ae3f3b19d40 scripts/rabbitmq-server
--- a/scripts/rabbitmq-server Fri Nov 14 17:41:03 2014 +0000
+++ b/scripts/rabbitmq-server Mon Nov 17 15:36:12 2014 +0000
@@ -94,8 +94,8 @@
${RABBITMQ_LISTEN_ARG} \
-sasl errlog_type error \
-sasl sasl_error_logger false \
- -rabbit error_logger '{file,"'${RABBITMQ_LOGS}'"}' \
- -rabbit sasl_error_logger '{file,"'${RABBITMQ_SASL_LOGS}'"}' \
+ -rabbit error_logger tty \
+ -rabbit sasl_error_logger tty \
-rabbit enabled_plugins_file "\"$RABBITMQ_ENABLED_PLUGINS_FILE\"" \
-rabbit plugins_dir "\"$RABBITMQ_PLUGINS_DIR\"" \
-rabbit plugins_expand_dir "\"$RABBITMQ_PLUGINS_EXPAND_DIR\"" \

and - somewhat irritatingly - it turns out that doing so causes the
management plugin to crash in 3.4.1. That bug will be fixed in 3.4.2
(and I'm pretty sure it was introduced in 3.4.0 since that's when we
started announcing log locations in mgmt).

You then have the slight problem that the native Erlang/RabbitMQ logging
doesn't put one event on a line, so I don't know how much sense systemd
can make of it.

Cheers, Simon

[Jeffrey Ollie]

Sorry to be dragging up an old thread, but I tried to make this change to my system and ended up with the following crash:

Feb 17 08:42:28 rabbitmq-server[24034]: BOOT FAILED
Feb 17 08:42:28 rabbitmq-server[24034]: ===========
Feb 17 08:42:28 rabbitmq-server[24034]: Error description:
Feb 17 08:42:28 rabbitmq-server[24034]: {error,{cannot_log_to_tty,sasl_report_tty_h,not_installed}}
Feb 17 08:42:28 rabbitmq-server[24034]: Log files (may contain more information):
Feb 17 08:42:28 rabbitmq-server[24034]: tty
Feb 17 08:42:28 rabbitmq-server[24034]: tty
Feb 17 08:42:28 rabbitmq-server[24034]: Stack trace:
Feb 17 08:42:28 rabbitmq-server[24034]: [{rabbit,ensure_working_log_handler,5,[{file,"src/rabbit.erl"},{line,711}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,ensure_working_log_handlers,0,
Feb 17 08:42:28 rabbitmq-server[24034]: [{file,"src/rabbit.erl"},{line,697}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,311}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,336}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {init,start_it,1,[]},
Feb 17 08:42:28 rabbitmq-server[24034]: {init,start_em,1,[]}]
Feb 17 08:42:28 rabbitmq-server[24034]: =INFO REPORT==== 17-Feb-2015::08:42:28 ===
Feb 17 08:42:28 rabbitmq-server[24034]: Error description:
Feb 17 08:42:28 rabbitmq-server[24034]: {error,{cannot_log_to_tty,sasl_report_tty_h,not_installed}}
Feb 17 08:42:28 rabbitmq-server[24034]: Log files (may contain more information):
Feb 17 08:42:28 rabbitmq-server[24034]: tty
Feb 17 08:42:28 rabbitmq-server[24034]: tty
Feb 17 08:42:28 rabbitmq-server[24034]: Stack trace:
Feb 17 08:42:28 rabbitmq-server[24034]: [{rabbit,ensure_working_log_handler,5,[{file,"src/rabbit.erl"},{line,711}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,ensure_working_log_handlers,0,
Feb 17 08:42:28 rabbitmq-server[24034]: [{file,"src/rabbit.erl"},{line,697}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,311}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,336}]},
Feb 17 08:42:28 rabbitmq-server[24034]: {init,start_it,1,[]},
Feb 17 08:42:28 rabbitmq-server[24034]: {init,start_em,1,[]}]
Feb 17 08:42:29 rabbitmq-server[24034]: {"init terminating in do_boot",{rabbit,failure_during_boot,{error,{cannot_log_to_tty,sasl_report_tty_h,not_installed}}}}
Feb 17 08:42:29 rabbitmq-server[24034]: Crash dump was written to: erl_crash.dump
Feb 17 08:42:29 rabbitmq-server[24034]: init terminating in do_boot ()

A little googling doesn't turn up much except that it's happened to others:

https://github.com/docker-library/rabbitmq/blob/master/Dockerfile

I'm not an Erlang guru, can someone point out how to fix this?

--

Jeff Ollie

[Simon MacMullen]

On 17/02/15 14:55, Jeffrey Ollie wrote:
> Sorry to be dragging up an old thread, but I tried to make this change
> to my system and ended up with the following crash:
>
> Feb 17 08:42:28 rabbitmq-server[24034]: BOOT FAILED
> Feb 17 08:42:28 rabbitmq-server[24034]: ===========
> Feb 17 08:42:28 rabbitmq-server[24034]: Error description:
> Feb 17 08:42:28 rabbitmq-server[24034]:
> {error,{cannot_log_to_tty,sasl_report_tty_h,not_installed}}

> A little googling doesn't turn up much except that it's happened to others:
>
> https://github.com/docker-library/rabbitmq/blob/master/Dockerfile
>
> I'm not an Erlang guru, can someone point out how to fix this?

It's not obvious to me what's happening. The point at which it's
crashing, it's attempting to swap out the default sasl log handler to
swap in tty, and finding it's not there.

I can't reproduce this, so can you tell me (in enough detail to
reproduce it) how you're installing and running RabbitMQ?

Cheers, Simon

[Jeffrey Ollie]

On Tue, Feb 17, 2015 at 11:46 AM, Simon MacMullen <si...@rabbitmq.com> wrote:

It's not obvious to me what's happening. The point at which it's crashing, it's attempting to swap out the default sasl log handler to swap in tty, and finding it's not there.

I can't reproduce this, so can you tell me (in enough detail to reproduce it) how you're installing and running RabbitMQ?

I'm running a custom built RabbitMQ RPM on Fedora 20, which has Erlang R16B03-1.  There's nothing fancy about the RPM, it's just the RPM from Rawhide rebuilt for Fedora 20 with a patch that I added to change the logging.  RabbitMQ version is 3.4.4.

--

Jeff Ollie

[Simon MacMullen]

On 17/02/15 17:55, Jeffrey Ollie wrote:
> I'm running a custom built RabbitMQ RPM on Fedora 20, which has Erlang
> R16B03-1. There's nothing fancy about the RPM, it's just the RPM from
> Rawhide rebuilt for Fedora 20 with a patch that I added to change the
> logging. RabbitMQ version is 3.4.4.

Hmm, now that I go back to look at it I don't understand how the
previous patch I posted ever worked.

Please try:

diff --git a/scripts/rabbitmq-server b/scripts/rabbitmq-server
index aea8bfc..0966080 100755
--- a/scripts/rabbitmq-server
+++ b/scripts/rabbitmq-server
@@ -93,9 +93,9 @@ exec ${ERL_DIR}erl \
${RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS} \

${RABBITMQ_LISTEN_ARG} \
-sasl errlog_type error \

- -sasl sasl_error_logger false \

- -rabbit error_logger '{file,"'${RABBITMQ_LOGS}'"}' \
- -rabbit sasl_error_logger '{file,"'${RABBITMQ_SASL_LOGS}'"}' \

+ -sasl sasl_error_logger tty \

+ -rabbit error_logger tty \
+ -rabbit sasl_error_logger tty \
-rabbit enabled_plugins_file "\"$RABBITMQ_ENABLED_PLUGINS_FILE\"" \
-rabbit plugins_dir "\"$RABBITMQ_PLUGINS_DIR\"" \
-rabbit plugins_expand_dir "\"$RABBITMQ_PLUGINS_EXPAND_DIR\"" \

Cheers, Simon

[Jeffrey Ollie]

On Fri, Feb 20, 2015 at 10:55 AM, Simon MacMullen <si...@rabbitmq.com> wrote:

On 17/02/15 17:55, Jeffrey Ollie wrote:

I'm running a custom built RabbitMQ RPM on Fedora 20, which has Erlang
R16B03-1.  There's nothing fancy about the RPM, it's just the RPM from
Rawhide rebuilt for Fedora 20 with a patch that I added to change the
logging.  RabbitMQ version is 3.4.4.

Hmm, now that I go back to look at it I don't understand how the previous patch I posted ever worked.

Ah yes, this new patch worked splendidly!  Thanks!

--

Jeff Ollie