Issues with rabbitmq federation

[Naina]

I am setting up federation on one rabbitmq broker (B) with another broker as the upstream broker (A).

I'm getting an error on trying to run the command to set up the upstream broker. The commands below were run on broker B.

localhost:$ sbin/rabbitmq-plugins enable rabbitmq_federation
The following plugins have been enabled:
  rabbitmq_federation
Plugin configuration has changed. Restart RabbitMQ for changes to take effect.

localhost:$ sbin/rabbitmq-plugins enable rabbitmq_federation_management
The following plugins have been enabled:
  rabbitmq_federation_management

localhost:$ sbin/rabbitmqctl set_parameter federation-upstream my-upstream '{"uri":"amqp://localhost:5672","expires":3600000}'
Setting runtime parameter "my-upstream" for component "federation-upstream" to "{\"uri\":\"amqp://localhost:5672\",\"expires\":3600000}" ...
Error: Validation failed

component federation-upstream not found

I did restart the broker after enabling the federation plugins. Why am I getting the error "component federation-upstream not found" ?

I could set up the upstream broker and policy through the Management UI, but federation is still not working.

[Michael Klishin]

Did you restart RabbitMQ after enabling the plugins, as rabbitmq-plugins suggested?

MK

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Naina]

Thanks for your reply. You're right, I had not restarted the upstream broker. I got past that, but federation is still not working.

localhost:$ sbin/rabbitmqctl set_parameter federation-upstream trial-upstream '{"uri":"amqp://localhost:5672","expires":3600000}'
Setting runtime parameter "trial-upstream" for component "federation-upstream" to "{\"uri\":\"amqp://localhost:5672\",\"expires\":3600000}" ... 

...done.

localhost:$ sbin/rabbitmqctl set_policy --apply-to exchanges federate-me "^subscription_exchange" '{"federation-upstream-set":"all"}'

Setting policy "federate-me" for pattern "^subscription_exchange" to "{\"federation-upstream-set\":\"all\"}" with priority "0" ...

...done.

localhost:$ sbin/rabbitmqctl eval 'rabbit_federation_status:status().'

[]

...done.

Federation Status page on the Management UI doesn't have any Running Links either. Is there anything else I need to do to get federation working? There have been no messages yet. Does that matter?

Thanks.

[Fredrik Sörensson (Fredrik Sorensson/STO/Fri]

Hello.

 

Does the policy match any exchanges? RMQ will not create links to any upstream queuemanagers unless there is something to actually federate.

 

In your case, if there are no exchanges with names matching the regular expression "^subscription_exchange”, there is nothing to federate, and the connections to the upstreams queuemanager will not be created.

 

On a personal note, I am not comfortable with using  "{\"federation-upstream-set\":\"all\"}". I usually do directed federations with a named set, or if I have a single upstream I use “federation-upstream”:”<upstream_name>” . In your case, that would be "{\"federation-upstream\":\" trial-upstream \"}", I think. But that is just me.

 

/Fredrik

[Naina]

Yes, there is an exchange called subscription_exchange in the upstream broker. I expect a corresponding exchange will be created downstream when federation starts working - is that right? Or do I have to create an exchange by the same name downstream?

Thanks for the tip about not using 'all', will restrict it once this starts working.

To post to this group, send email to rabbit...@googlegroups.com.

[Simon MacMullen]

On 15/01/2015 18:39, Naina wrote:
>
> Yes, there is an exchange called subscription_exchange in the upstream
> broker. I expect a corresponding exchange will be created downstream
> when federation starts working - is that right?

No, it's the other way round.

Federation needs to be set up on the downstream, and any exchange needs
to be created by you on the downstream.

If no corresponding exchange exists on the upstream, it will then be
created.

Cheers, Simon

[Naina]

Thanks. Federation is working now. I bound a queue to the federated exchange. The queue exists on the upstream broker too. Consumers are currently there only for the upstream broker.

When a message was published using a connection factory associated with the upstream broker, I could see the message as "ready" in the management UI for the downstream broker. 

I do not see the message on the upstream broker's management UI.

However the message got consumed successfully, which is confusing because there are no consumers set up on the downstream broker.

Questions:

Is it normal that there is no sign on the message on the upstream broker although it was published to the upstream broker?

Why did it get consumed?

[Michael Klishin]

On 26 January 2015 at 23:11:18, Naina (shalmali...@gmail.com) wrote:
> However the message got consumed successfully, which is confusing
> because there are no consumers set up on the downstream broker.
>
> Questions:
> Is it normal that there is no sign on the message on the upstream
> broker although it was published to the upstream broker?
>
> Why did it get consumed?

Exchange federation should generally not affect queues bound to the upstream exchange
(unless you use a consistent hashing exchange or random exchange or other types where the # of queues
bound may affect routing).

Without knowing how your topology is set up (specific exchange and queue names, exchange types,
and bindings) we cannot suggest much. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Naina]

In more general terms, in case of a federated exchange would messages get consumed by both upstream and downstream brokers (assuming queue and consumer set up is the same on both brokers)? 

To prevent that do I make the queues federated so the messages go to either the upstream or the downstream broker?

Can federation be set up in such a way that there is not single point of failure? The upstream broker seems to be the single point of failure here. And if multiple upstream brokers are used in a clustered fashion then they have to be in the same region, which is again a risk.

I'm trying to achieve high availability here and exploring federation as an alternative to clustering, since our brokers are across a WAN. I have read up on most of the documentation on federation but some questions are still unanswered.

Thanks for your help.

[Michael Klishin]

On 27 January 2015 at 00:41:34, Naina (shalmali...@gmail.com) wrote:
> In more general terms, in case of a federated exchange would
> messages get consumed by both upstream and downstream brokers
> (assuming queue and consumer set up is the same on both brokers)?
> To prevent that do I make the queues federated so the messages
> go to either the upstream or the downstream broker?

Please take a look at http://www.rabbitmq.com/federated-exchanges.html#details.

> Can federation be set up in such a way that there is not single point
> of failure? The upstream broker seems to be the single point of
> failure here. And if multiple upstream brokers are used in a clustered
> fashion then they have to be in the same region, which is again
> a risk.
> I'm trying to achieve high availability here and exploring federation
> as an alternative to clustering, since our brokers are across
> a WAN. I have read up on most of the documentation on federation
> but some questions are still unanswered.
> Thanks for your help.

To me it seems to be quite similar to how a single publisher can be SPoF. You can have
multiple upstreams to a single downstream (just like you can have multiple publishers).

The key trade-off is that with redundant publishers is that your consumers will need
to be prepared to handle duplicates.