rabbitmq memory usage

78 views
Skip to first unread message

Sergey Kraev

unread,
Feb 25, 2014, 8:38:13 AM2/25/14
to rabbitmq...@googlegroups.com

Hi, all!

I am stress testing RabbitMQ.

I have 10 persistent queues without consumers which I start to fill from topic exchange with 100 bytes long messages at the fastest rate possible. One hour and 100 million messages later erlang process is using all available RAM (3 GB) and message publishing rate at each queue drops from 3000 messages per second to zero.

I’d like to know if this behavior is expected and for given configuration the queue backlog limit is 100 million messages.

Thanks!

Alvaro Videla

unread,
Feb 25, 2014, 10:43:37 AM2/25/14
to Discussions about RabbitMQ, rabbitmq...@googlegroups.com
Hi,

Have you seen http://www.rabbitmq.com/memory.html ?

Regards,

Alvaro
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq...@lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>

the...@gmail.com

unread,
Feb 25, 2014, 1:41:34 PM2/25/14
to rabbitmq...@googlegroups.com, Discussions about RabbitMQ
Thanks for the link!

So the publishing rate drops to zero, because Erlang RAM usage goes over the limit of vm_memory_high_watermark (in my case, 3GB).

Then my question is: is it normal that 100 million persisted messages use 3 GB of memory? Shouldn't broker page them to disk and free memory? After I hit the limit I waited for an hour, and memory usage didn't go down and connections remained blocked.

Regards,
Sergey



--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-discuss/0YGp-fpNk3Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-discu...@googlegroups.com.
To post to this group, send email to rabbitmq...@googlegroups.com.
Visit this group at http://groups.google.com/group/rabbitmq-discuss.
For more options, visit https://groups.google.com/groups/opt_out.

the...@gmail.com

unread,
Feb 26, 2014, 2:45:15 AM2/26/14
to rabbitmq...@googlegroups.com, Discussions about RabbitMQ
This is memory usage breakdown:

{memory,
[{total,3485934240},
{connection_procs,34710096},
{queue_procs,138552056},
{plugins,436080},
{other_proc,19191352},
{mnesia,107120},
{mgmt_db,598560},
{msg_index,2829261600},
{other_ets,425229944},
{binary,12478704},
{code,19777382},
{atom,703377},
{other_system,4887969}]},



On Tue, Feb 25, 2014 at 5:43 PM, Alvaro Videla <videl...@gmail.com> wrote:
Reply all
Reply to author
Forward
0 new messages