Request: more control over queue naming / exchange binding via rabbitmq-stomp

[ni...@bluejeansnet.com]

I made some comments on rabbitmq-stomp#24, but it was rightfully suggested to be more clear and complete here on the list.

We have a use case today in which everyone is connecting via AMQP, but we want to allow end users with more rigid firewall rules (e.g., only able to make outbound https connections on port 443) to use our service as well.  We'd like to do that using the rabbitmq-web-stomp plugin and the /stomp/websocket endpoint.

Today we partition users with rabbitmq permissions such that they can read from exchanges and queues that begin with their username followed by a delimiter; for example, "enduser$exchange.name."  This works well with AMQP.  (We didn't use vhosting because we needed one 'super' account to be able to easily read and write anything for any username.)

For each end user, we have a pre-declared exchange that we know will always exist for them, "username$listener".  To this we bind predictably-named queues using routing keys.  (A nice capability of doing it this way is that if the end user needs to add capacity to their ability to handle incoming messages, they can add more instances binding to the same predictably-named queues.)  These queues are auto-delete, non-exclusive queues that we use only so long as something is connected and working.  We'd prefer not to pre-declare the queues, as we'd rather drop the messages on the floor if nothing is there to listen.

It looks like subscribing to /exchange/exchange-name/routing-key would almost give us what we want; however, this auto-generates a queue name in the typical amq.* format; because of our permissions scheme, the web-stomp client is then not able to read from this autogenerated queue.  I'd rather avoid modifying our permissions scheme to permit users to read from all temporary queues, and in general, I prefer to explicitly name queues so I can tell from the Admin plugin what a queue is for and where it's going at a glance.

One potential way this problem could be solved is allowing some x-header on the SUBSCRIBE message to specify a queue name when subscribing to an exchange with a routing key.  We already know what we want the queue name to be on our end (because we already do it this way in the AMQP case).  But we'd also need to potentially allow multiple endpoints to be able to subscribe to this same queue name-- we'd also need to be able to specify that it shouldn't be an exclusive queue.

Maybe if we could do something like "X-queue-name: temporary-queue-name" and "X-queue-exclusive: false" we could duplicate the behavior of the way we use AMQP today, but I'm certainly open to other ideas as well.

It might also work to look at the problem from the other direction and subscribe to /queue/queue-name, but be able to specify "X-exchange-name" and "X-routing-key" for the subscription instead of the default exchange with the queue name as the routing key.

Also a question that isn't entirely clear to me from looking at the docs for rabbitmq-stomp: there's a /amq/queue endpoint for accessing pre-declared queues that are managed by AMQP, and we can use that very nicely if we are in a position to create queues on the AMQP side.  Does the /exchange/exchange-name endpoint "see" messages from the AMQP side of things too, or do exchanges and queues handled by rabbitmq-stomp exist in an isolated universe from the rest of AMQP?

Thanks for any input, ideas, and suggestions!

    Nick

[Giuseppe Privitera]

yes, you can setup your queues from AMQP.

 Does the /exchange/exchange-name endpoint "see" messages from the AMQP side of things too, or do exchanges and queues handled by rabbitmq-stomp exist in an isolated universe from the rest of AMQP?

/exchange destinations are accessible from both AMQP and STOMP

Thanks for any input, ideas, and suggestions!

    Nick

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kind regards,

Giuseppe Privitera

[Nick Johnson]

On Wed, Sep 30, 2015 at 10:27 AM, Giuseppe Privitera <gpriv...@pivotal.io> wrote:

On 30 Sep 2015, at 17:38, ni...@bluejeansnet.com wrote:

Also a question that isn't entirely clear to me from looking at the docs for rabbitmq-stomp: there's a /amq/queue endpoint for accessing pre-declared queues that are managed by AMQP, and we can use that very nicely if we are in a position to create queues on the AMQP side.

yes, you can setup your queues from AMQP.

That's my backup plan if we can't come up with some finer-grain control (via x-headers or other means) to manage queue naming when subscribing to an exchange... that is, having a consumer sitting on our server consuming from an exchange specifically for requesting creation of the queues on-demand that our web-stomp end users can then send to (and then wait for some kind of reply or receipt confirming queue creation before subscribing to /amq/queue/new-queue-name).  It feels a bit clunky to do it this way, though, because it moves the queue creation away from the thing that wants to create the queues.

   Nick 

[Giuseppe Privitera]

We will look into it.

Kind regards,

Giuseppe Privitera

[ni...@bluejeansnet.com]

Should I open a RFE issue for rabbitmq-stomp?  I see that the one on which I was commenting (#24) is closed.

   Nick

[Michael Klishin]

Discussions of features (as opposed to implementation) belong to this list.

[Michael Klishin]

+rabbitmq-users

On 6 Oct 2015 at 18:41:12, ni...@bluejeansnet.com (ni...@bluejeansnet.com) wrote:
> Okay, I don't want to mess with your system :-)
>
> Please let me know if there's anything I can do to help along the
> way.

Well, we’d appreciate some specific ideas about how naming
can be controlled. We would rather not introduce our own pattern languages
and so on.

I can think of two ways:

 * Using regular expressions and replacement. The key question is “what values should be used for replacement”.
 * Using Erlang functions. This would require building a tiny custom plugin to
   make them available to the server, so it’s probably not for most.

In other words, we need to discuss specific ideas about how to make this possible. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Nick Johnson]

On Tue, Oct 6, 2015 at 11:51 AM, Michael Klishin <mkli...@pivotal.io> wrote:

Well, we’d appreciate some specific ideas about how naming
can be controlled. We would rather not introduce our own pattern languages
and so on.

I can think of two ways:

 * Using regular expressions and replacement. The key question is “what values should be used for replacement”.
 * Using Erlang functions. This would require building a tiny custom plugin to
   make them available to the server, so it’s probably not for most.

In other words, we need to discuss specific ideas about how to make this possible. 

For our use case, the best thing would be for us to be able to explicitly specify what we'd like the queue name to be when we subscribe to the topic exchange, since historically (from our AMQP connector) this has always been a known, predictable value that we can generate when we connect.

So if we could do something like:

SUBSCRIBE /topic/fooexchange

exclusive=false

auto_delete=true

x_queue_name=walter$0f3ec8405244482292ded091854796f4#foo

And then have rabbitmq-stomp create the queue name using our x-queue-name header instead of using the random, autogenerated name, it would give us exactly what we need.

In the case that the queue already existed and was already bound to the topic, it would be perfectly fine to reuse it.  If it was bound to some other topic, it would be OK with us to call that an error condition.

   Nick

[Michael Klishin]

On 6 Oct 2015 at 22:45:09, Nick Johnson (ni...@bluejeansnet.com) wrote:
> So if we could do something like:
>
> SUBSCRIBE /topic/fooexchange
> exclusive=false
> auto_delete=true
> x_queue_name=walter$0f3ec8405244482292ded091854796f4#foo
>
> And then have rabbitmq-stomp create the queue name using our
> x-queue-name header instead of using the random, autogenerated
> name, it would give us exactly what we need.
>
> In the case that the queue already existed and was already bound
> to the topic, it would be perfectly fine to reuse it. If it was bound
> to some other topic, it would be OK with us to call that an error
> condition.

This sounds reasonable. Please file a rabbitmq/rabbitmq-stomp issue, now that
we understand the scope and purpose of this feature .