RabbitMQ queues leaking memory

[Cristian Datculescu]

Hello all.

We have a small issue with RabbitMQ 3.6.10 (single node setup, docker container with volumes and network_mode host). The issue manifests itself like this:

We put a lot of messages in a queue with no consumers. When we purge the queue, the memory remains the same and only gets cleared out when the queue gets removed.

Is there any workaround against this?

Best,

Cristian.

[Michael Klishin]

You can see a memory usage breakdown on the node page in the management UI or in `rabbitmqctl status`,

and more detailed breakdown in rabbitmq-top.

Very likely this is just an effect of a lazy garbage collection of binaries in Erlang. RabbitMQ nodes force

GC at various points in the lifecycle.

You can enable periodic forced GC for all "idle" queues:

https://github.com/rabbitmq/rabbitmq-server/blob/stable/docs/rabbitmq.config.example#L336

If you expect queues without consumers to be a common scenario in your system, perhaps you want

to make those queues lazy: http://rabbitmq.com/lazy-queues.html.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Cristian Datculescu]

Awesome, thank you very much Michael for taking the time. I was not familiar with all the changes in RabbitMQ 3.6 branch (we are mostly running 3.5.6 branch).

Best,

Cristian.

On Monday, July 31, 2017 at 3:44:30 PM UTC+2, Michael Klishin wrote:

You can see a memory usage breakdown on the node page in the management UI or in `rabbitmqctl status`,

and more detailed breakdown in rabbitmq-top.

Very likely this is just an effect of a lazy garbage collection of binaries in Erlang. RabbitMQ nodes force

GC at various points in the lifecycle.

You can enable periodic forced GC for all "idle" queues:

https://github.com/rabbitmq/rabbitmq-server/blob/stable/docs/rabbitmq.config.example#L336

If you expect queues without consumers to be a common scenario in your system, perhaps you want

to make those queues lazy: http://rabbitmq.com/lazy-queues.html.

On Mon, Jul 31, 2017 at 4:35 PM, Cristian Datculescu <cristian....@gmail.com> wrote:

Hello all.

We have a small issue with RabbitMQ 3.6.10 (single node setup, docker container with volumes and network_mode host). The issue manifests itself like this:

We put a lot of messages in a queue with no consumers. When we purge the queue, the memory remains the same and only gets cleared out when the queue gets removed.

Is there any workaround against this?

Best,

Cristian.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

In 3.5.6 background GC is always enabled. Its interval self-tunes

to be every 60 seconds where possible.

Please consider upgrading to 3.6.11 when it goes GA this week (both because we

force binary GC in more cases under load and because you are ~ a dozen of release behind in general, including security patches).

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Cristian Datculescu]

Hi Michael and thanks again.

We have migrated one of the main clusters and we enabled gc for now. thanks for the hints of what is coming up in 3.6.11, we will give it a try (generally we like to leave a 1-2 months in between releases to make sure there is nothing wrong with the new version).

Best,

Cristian.