[rabbitmq-discuss] STOMP - Message queue selectors for consumers?

[Toby Corkindale]

Hi,
I'm afraid I'm something of a beginner when it comes to message queuing
systems, although I've submitted several patches to the Perl STOMP
modules in the process of using them recently, and have picked up what I
think is a bit of knowledge.

Anyway, I have a query about RabbitMQ's STOMP plugin, and haven't quite
been able to understand the docs on the matter.

My goal is to have a queue where the consumers of that queue only
receive a subset of all messages. I want them to filter based on a
header in the messages.

Eg.

SEND
destination:/queue/foo
x-country:australia

with the receiver somehow doing something like:
SUBSCRIBE
destination:/queue/foo?x-country=australia

Now currently I can achieve this goal by creating lots of queues, one
for each possible value, and then the consumers can subscribe to all the
queues.. but this is a bit messy.

Eg.
SEND
destination:/queue/foo.australia

SUBSCRIBE
destination:/queue/foo.australia

There are some discussions of routing keys in the STOMP plugin's docs,
but I don't understand how they relate. Are they what I'm looking for?
I only want this to work with simple queues, not, as far as I know,
exchanges. Maybe I'm barking up the wrong tree though?

I note that in ActiveMQ's STOMP plugin, you can apply a selector to the
subscription like this:

SUBSCRIBE
destination:/queue/foo
activemq.selector:country="australia"

Thanks,
Toby
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Michael Justin]

This is also documented in a broker independent way on
http://stomp.codehaus.org/Stomp+JMS

"You can use an SQL 92 selector whenever you subscribe using the
selector header

selector:location = 'London' and sex = 'M'

It is also supported by the Oracle Open Message Queue (OpenMQ) Stomp
connector.

But the RabbitMQ Stomp plugin does not support the JMS+Stomp extensions
like no-local, expires, persistent, priority, reply-to or
durable-subscriber-name etc. AFAIK - but please correct me if I am wrong :)

Regards,
Michael Justin

[Rob Harrop]

Toby,

The solution you propose is the easiest and most efficient way of doing it on RabbitMQ STOMP. Using different /queue/ destinations removes the need for complex matching during message routing.

Queues are cheap in RabbitMQ and each /queue corresponds to one queue, so creating a queue per country is not expensive. Also, queues are created lazily so you don't need to worry about ordering consumers/publishers.

There is another option that you could consider that has _slightly_ different semantics: use a destination such as /exchange/amq.direct/country.australia.

The main difference between /queue and /exchange in this instance is how multiple subscribers are treated.

With /queue, if you have two subscribers to the same country queue, then messages are sent out to the consumers in a round-robin fashion.

With /exchange, each subscriber will get a copy of each message. 

I'm guessing for your use case that round-robin is probably the most suitable.

Regards,

Rob

-- 
Rob Harrop
RabbitMQ

[Rob Harrop]

But the RabbitMQ Stomp plugin does not support the JMS+Stomp extensions
like no-local, expires, persistent, priority, reply-to or
durable-subscriber-name etc. AFAIK - but please correct me if I am wrong :)

We do support the 'persistent' header, and you're free to add in reply-to headers on any SEND frame and they will be present in the corresponding MESSAGE frames. 

[Michael Justin]

Am 07.06.2011 13:36, Rob Harrop wrote:

> We do support the 'persistent' header, and you're free to add in
> reply-to headers on any SEND frame and they will be present in the
> corresponding MESSAGE frames.

Thank you for the correction! I have tested it in my Habari RabbitMQ
Client library, and it works as expected. Also the performance is very
impressive, I can easily exchange up to 3000 messages per second (send
and receive) on an entry-level system with the client running on the
same system as RabbitMQ 2.4.1

[Toby Corkindale]

On 07/06/11 21:30, Rob Harrop wrote:
> Toby,
>
> The solution you propose is the easiest and most efficient way of doing
> it on RabbitMQ STOMP. Using different /queue/ destinations removes the
> need for complex matching during message routing.
>
> Queues are cheap in RabbitMQ and each /queue corresponds to one queue,
> so creating a queue per country is not expensive. Also, queues are
> created lazily so you don't need to worry about ordering
> consumers/publishers.

Hi Rob,
Thanks, that's good to know.

Let's say I change the example to using machine hostnames or IP
addresses. In that case there are not going to be large numbers of
unique names in use at any one time, but over time the names may stop
being used and new ones will appear.

So I could use STOMP destinations such as:
/queue/hostname.foobar.example.com

Is there a way to clean up the queues once those machines have gone
away, say after some months or more of use?

I couldn't spot anything on the STOMP plugin page, nor the rabbitmq
admin guide page.

I can see there is a "list_queues" option to rabbitmqctl, but not a
corresponding delete_queue option.. Where would I find that?

Sorry about all these newbie questions..
Thanks again for your help,

[Rob Harrop]

Tony,

The best option here is to use the management plugin (http://www.rabbitmq.com/management.html).

You'll be able to manually delete the queues using the UI or script the deletion using the REST API or the rabbitmqadmin plugin.

If you think it would be useful we can add support to STOMP for queue leases (http://www.rabbitmq.com/extensions.html#queue-leases).

-- 
Rob Harrop
RabbitMQ