How to deal with a lot of *.rdq files (68 GB) in \msg_store_transient\ safely? How to prevent it?

[SlavaZ]

Hello All,

Our partners had the following issue:

"We had an issue: one of the rabbit folders is having more than 68 GB of data. 

The folder is: 

...\AppData\Roaming\RabbitMQ\db\rabbit@server-mnesia\msg_stores\vhosts\628WB79CIFDYO9LJI6DKMI09L\msg_store_transient

In my lab, this folder has one file.

We had to reinstall the rabbit because the hard drive is full. 

We need a way to prevent these files to be so big."

They provided a screenshot where a lot of .rdq files (16 MB each) exist and they have the same date/time stamp - three weeks ago.

However, the partners indicate the issue appeared last week.

I doubt the issue with 68 GB of data did appear at once. 

Could you please comment on this?

1) As I understand these files do not belong to an existing queue, in this case, the files will be located under the \queues\ folder. Is this correct?

2) Is there a recommended way to clean up the folder \msg_store_transient\ from *.rdq files safely?

3) How to prevent such issues in the future? 

Is there a way to monitor related markers in RabbitMQ Web Management console?

4) How to restore the functionality of RabbitMQ next time when this issue appears (huge amount of .rdq files)?

Thank you.

Configuration:

 RabbitMQ 3.7.4, No RabbitMQ Federation.

 Erlang 19.3

 Windows Server 2016

No Logs, unfortunately.

[Philip Kuryloski]

Hi,

*.rdq files store messages in queues on disk. So, generally speaking, limiting the number of backlogged messages in your RabbitMQ should serve to limit the amount of space required. Queue length limits https://www.rabbitmq.com/maxlength.html can be useful in this regard as well. If disk space started to be consumed rapidly and recently, it is worth checking if something has happened to your consumers.

Without logs there isn't much more I can say. Erlang 19.3 is quite old, as is RabbitMQ 3.7.4, so I would encourage you to upgrade if all all possible.

-Phil