Ability to filter or limit messages > x bytes

[Anders Malm]

Hi

Is RabbitMQ able to filter out or reject messages that are over a certain size limit (32766 bytes)? I can only find limitations pertaining to the queue itself in this kb: https://www.rabbitmq.com/maxlength.html

Even re-routing messages to another queue based on their size would be an option.

Any advice is appreciated.

TIA

/Anders

[Wesley Peng]

There might not be customized messages size limitation in broker, but you could do it in the client.

What cloudamqp says,

While the theoretical message size limit in RabbitMQ is 2GB up to 3.7.0, we don't recommend sending messages larger than 128MB, which is also the new max size limit in 3.8.0 and onward. Large messages are especially problematic when using mirrored queues in HA clusters and can cause memory and performance issues.

Regards 

Wesley Peng

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/a9daac64-129b-4b37-9d44-054a567beac6%40googlegroups.com.

[Luke Bakken]

Hi Anders,

RabbitMQ does support a global max message size. You can configure it in rabbitmq.conf with this setting:

max_message_size = 32KiB

If you're using rabbitmq.config or advanced.config, the setting is:

[

    {rabbit, [

        {max_message_size = 32768}

    ]}

].

The above would cause any message greater than that size to be rejected. The default for this value is 128MiB.

https://github.com/rabbitmq/rabbitmq-server/pull/1812

This setting isn't documented as adjusting it is expected to be rare.

Thanks,

Luke

[Anders Malm]

Hi Luke

Thanks for your input, I'll give it a shot and report back. :)

BR. Anders Malm

[Anders Malm]

Hi Luke

Just attempted this in our dev environment, but ran into an error when restarting the service.

11:16:26.460 [error]    You've tried to set max_message_size, but there is no setting with that name.
11:16:26.460 [error]    Did you mean one of these?
11:16:26.485 [error]    handshake_timeout
11:16:26.485 [error]    log.upgrade.file
11:16:26.485 [error]    delegate_count
11:16:26.485 [error]    Error generating configuration in phase transform_datatypes
11:16:26.485 [error]    Conf file attempted to set unknown variable: max_message_size

We are currently running RabbitMQ version 3.7.15 on our servers. Is this too old for the max_message_size parameter?

TIA

Anders Malm

On Monday, December 2, 2019 at 9:41:10 PM UTC+1, Luke Bakken wrote:

[Stuart Johnston]

I think that function is only available in rabbitMQ v3.8

[Anders Malm]

Updating the cluster to 3.8.2 solved not being able to start the service with the max_message_size parameter.

I did have to change the example from Luke to max_message_size = 32768 rather than max_message_size = 32KiB.

Thanks

Anders Malm