RabbitMQ incorrectly detect available memory when deployed in Docker container

[puneet nagpal]

Hi,

I am using totem- RabbitMQ version 3.6.12 with erlang/OTP 20.0 for IOT purpose and using MQTT and have deployed in docker container

Configuration:

Host Memory configuration (where container is deployed and running):256 GB RAM

Container allowed Memory (RAM) : 16 GB (defined using marathon)

vm_memory_high_watermark set to 0.1, so ideally rabbitMQ should reserver 10% of 16GB which 1.6 GB

what i observed is that when rabbitMQ starts then it shows (checked using rabbitmq logs) that 10% of 256 = 25.6 GB

Instead of taking Memory from container (10% of 16GB) using /sys/fs/cgroup/memory/memory.limit_in_bytes

it is taking from HOST kernel ?

Is this as per the Design or bug or am i missing something in configuration ?

Regards,

Puneet

[Michael Klishin]

RabbitMQ retrieves total amount of memory available from the kernel and does not consult /sys/fs/cgroup.

In 3.6.12 you can override it using

`rabbit. total_memory_available_override_value `: https://github.com/rabbitmq/rabbitmq-server/blob/stable/docs/rabbitmq.config.example#L264.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[puneet nagpal]

Thanks MK for quick reply, 

Couple of questions

1)  can i set directly 1.6GB as total_memory_available_override_value ?,

Is there any limitation here while setting this (as i am using 10% of available memory and recommended is .4 to .66%) ? ofcourse i will not go beyond kernal memory

and will ensure that

• Nodes hosting RabbitMQ should have at least 128MB of memory available at all times.

2). what is mem_relative mean while setting disk_free_limit ?, is it relative to RAM allocated to RabbitMQ node ?

Regards,

Puneet

On Wednesday, September 13, 2017 at 11:07:35 AM UTC-4, Michael Klishin wrote:

RabbitMQ retrieves total amount of memory available from the kernel and does not consult /sys/fs/cgroup.

In 3.6.12 you can override it using

`rabbit. total_memory_available_override_value `: https://github.com/rabbitmq/rabbitmq-server/blob/stable/docs/rabbitmq.config.example#L264.

On Wed, Sep 13, 2017 at 11:02 AM, puneet nagpal <puneetn...@gmail.com> wrote:

Hi,

I am using totem- RabbitMQ version 3.6.12 with erlang/OTP 20.0 for IOT purpose and using MQTT and have deployed in docker container

Configuration:

Host Memory configuration (where container is deployed and running):256 GB RAM

Container allowed Memory (RAM) : 16 GB (defined using marathon)

vm_memory_high_watermark set to 0.1, so ideally rabbitMQ should reserver 10% of 16GB which 1.6 GB

what i observed is that when rabbitMQ starts then it shows (checked using rabbitmq logs) that 10% of 256 = 25.6 GB

Instead of taking Memory from container (10% of 16GB) using /sys/fs/cgroup/memory/memory.limit_in_bytes

it is taking from HOST kernel ?

Is this as per the Design or bug or am i missing something in configuration ?

Regards,

Puneet

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

1. Any positive value will be  accepted.

2. Please take a look at alarm docs.

--

Staff Software Engineer, Pivotal/RabbitMQ

[Sohail Ahmed]

I think you misunderstood the purpose of vm_memory_high_watermark.
The default is 0.4 which means RabbitMQ will stop accepting messages upon reaching 40% of memory usage, so it is reserving 60% for its use.

Memory Alarms
https://www.rabbitmq.com/memory.html

Sohail