Policies with queue max-length and lazy mode

[4integ...@gmail.com]

Hi,

I have a policy with regex ^TEST\.\w+ applies to queues and exchanges with the following definition:

ha-mode: all

ha-sync-mode: automatic

max-length: 10000

queue-mode: lazy

When creating a queue TEST.MYQUEUE neither max-length or queue-mode is set on the queue.

Shouldn't it be?

/ Joacim

[Michael Klishin]

We don't have enough information to tell. Specific steps to reproduce and a definition export file would help a lot.

By far the most common issue with policies that are not applied comes down to one often overlooked aspect:

only ONE policy is applied to any given queue or exchange at a time.

if there's already an existing policy that matches the same queue, consider that the effective policy is picked randomly. If this

is not OK (and usually it is not), policies must be assigned explicit priorities. All of this should be covered in [1].

While at it, it is a good idea to separate queue and exchange policies. Exchanges won't be affected by properties

such as queue-mode or max-length but it won't make any easier to investigate things for your future self (and your coworkers).

1. http://www.rabbitmq.com/parameters.html#policies

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[4integ...@gmail.com]

Hi Michael,

Yes, I have read about the one policy being used and priorities. 

I am using RabbitMQ 3.6.10 and Erlang 19.3

Attached the definition export (only with few changes for anonymity :) )

/ Joacim

On Thursday, October 5, 2017 at 11:03:36 PM UTC+2, Michael Klishin wrote:

We don't have enough information to tell. Specific steps to reproduce and a definition export file would help a lot.

By far the most common issue with policies that are not applied comes down to one often overlooked aspect:

only ONE policy is applied to any given queue or exchange at a time.

if there's already an existing policy that matches the same queue, consider that the effective policy is picked randomly. If this

is not OK (and usually it is not), policies must be assigned explicit priorities. All of this should be covered in [1].

While at it, it is a good idea to separate queue and exchange policies. Exchanges won't be affected by properties

such as queue-mode or max-length but it won't make any easier to investigate things for your future self (and your coworkers).

1. http://www.rabbitmq.com/parameters.html#policies

On Thu, Oct 5, 2017 at 12:09 PM, <4integ...@gmail.com> wrote:

Hi,

I have a policy with regex ^TEST\.\w+ applies to queues and exchanges with the following definition:

ha-mode: all

ha-sync-mode: automatic

max-length: 10000

queue-mode: lazy

When creating a queue TEST.MYQUEUE neither max-length or queue-mode is set on the queue.

Shouldn't it be?

/ Joacim

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[4integ...@gmail.com]

Hi,

Any input on this?

I just upgraded to 3.6.12 but no change in behavior.

/ Joacim

[Michael Klishin]

Hi,

All policies in the definitions file have the same priority. I suspect that my hypothesis above is correct.

Only one policy is applied to a queue at a time. if there's already an existing policy that matches the same queue,

consider that the effective policy is picked randomly. If this
is not OK (and usually it is not), policies must be assigned explicit priorities. All of this should be covered in [1].

In case it is not obvious what you should do: either use policy priorities or combine all the keys you want into a single policy.
There are no other options (in 3.6.x anyway) and this is not a bug.

1. http://www.rabbitmq.com/parameters.html#policies

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[4integ...@gmail.com]

Hi Michael,

Thanks for your reply.

What I understand is happening:

1. Get the matching policies for a queue based on queue name and RegEx defined on policy

2. Of the matches - pick the one with highest priority. If more than one match exist, pick one randomly.

In my case I have divided the policies into separate ones for queues and exchanges (as you recommended) and it looks as in screenshot below

When creating a queue CRM.SOMETHING.QUEUE I can see the correct and assumed policy is applied (see screenshot below).

But the "features" like queue-mode=lazy and max-length=10000 aren't applied BUT the ha-mode/ha-sync-mode are applied

Am I missing something here ?

/ Joacim

[Luke Bakken]

Hi Joacim -

Have you sent messages to your CRM.* queues? The reason I ask is that I imported your definitions locally, published messages, and saw that message count was limited to 10,000 and that those messages were stored on disk (i.e. lazy mode).

You are correct that the queues page does not reflect these parts of the policy and thus requires you to investigate the policy yourself. But, if you publish messages to the queues, you can see that both queue-mode: lazy and max-length: 10000 are applied.

Could you try this out in your environment and let me know how it works?

I have created an issue to address this here if you have a GitHub account and would like to follow along: https://github.com/rabbitmq/rabbitmq-management/issues/492

Thanks,

Luke

[V Z]

I think Joacim is expecting queue features to reflect limits expressed in the policy, but that is not what is happening. Queue features only show limits applied to the queue directly, not via the policy. Even though queue features do not reflect limits set by the policy, limits set by the policy will take effect (per my understanding and experience).

[4integ...@gmail.com]

Hi Luke,

Yes, you are correct!

The "features" is indeed applied and used through policies but not visible more than through reading policies and manually verifies the queues.

To have a consistent view I think it would be of great benefit to see queue features on the queue page - both explicit defined and implicit through policies.

Thanks for the reported issue!

/ Joacim

[V Z]

I second that. Clearly seeing what constraints are applied to the policy would be very helpful, especially when the policy is in conflict with the queue arguments.