Rabbitmq message limit per second

[Kostas Karkaletsis]

I am using RabbitMQ 3.7.4 for testing. I try to stress test the service and I have create a test script that generates 3000 messages to be sent to Rabbit.

When I see the logs in the server I see the following
2018-04-25 12:59:34.003 [warning] <0.32.0> lager_error_logger_h dropped 141 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:35.003 [warning] <0.32.0> lager_error_logger_h dropped 220 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:36.003 [warning] <0.32.0> lager_error_logger_h dropped 72 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:37.040 [warning] <0.32.0> lager_error_logger_h dropped 134 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:38.010 [warning] <0.32.0> lager_error_logger_h dropped 36 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:40.001 [warning] <0.32.0> lager_error_logger_h dropped 42 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:41.026 [warning] <0.32.0> lager_error_logger_h dropped 26 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:43.000 [warning] <0.32.0> lager_error_logger_h dropped 30 messages in the last second that exceeded the limit of 1000 messages/sec
2018-04-25 12:59:44.124 [warning] <0.32.0> lager_error_logger_h dropped 47 messages in the last second that exceeded the limit of 1000 messages/sec

Is there somewhere a setting of increasing / configuring the limit of 1000messages/sec

[Michael Klishin]

There is a way to configure that in Lager [1], the logging library RabbitMQ uses starting with 3.7:

``` erlang
[
  {lager, [
    {error_logger_hwm, 300}
  ]}
].
```

(the default is 50). It can only be configured via the advanced.config file (or in the classic rabbitmq.config).

I could swear this list had this question a couple of times before but group search yields no results :/

RabbitMQ never logs messages so whatever your test does, it's something else that stresses the logger (possibly a high connection churn or channel exception rate).

1. https://github.com/erlang-lager/lager/blob/6cc6b125ad55835748cd847f03314698ae439aea/src/lager.app.src#L50

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Matteo Spreafico]

Hi,

I know this thread is a bit old now, but I'm noticing the very same behavior. The solution you propose is the same I've read in other posts, but there's something I can't understand and nobody answers:

where are the NON-dropped error messages? I, and other users too, don't see high error rates in logs.

thanks

Matteo

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Luke Bakken]

Hi Matteo,

That's a good question. I wonder if the other messages are at a log level that isn't going to the file (debug, for instance).

I'll do some testing to find out.

Thanks,

Luke

[Luke Bakken]

Hi Matteo,

Please see this message: https://groups.google.com/d/msg/rabbitmq-users/zLo3KdKn-hw/rQZ_cBmhCQAJ

A fix is in the works.

Thanks,

Luke

[Luke Bakken]

Hello,

The following fix has been merged: https://github.com/erlang-lager/lager/pull/499

Expect to see this ship with RabbitMQ 3.7.15

Thanks,
Luke