Rabbitmq cluster switching slave to a master

[Mark Wick]

I have a 2 node cluster with HA queue mirroring.

I use "rabbitmqctl report" on the "slave" and see a bunch of queues for the vhosts we have defined

I stop the master

I run the "rabbitmqctl report" again and see nothing, always

Shouldn't I be seeing the mirror'd queues on the slave?  And shouldn't those now become anchored on the slave whereby the slave is now the master?

I guess I'm just a bit confused what may be going on here.

NOTE:  Rabbit version is 3.3.1 with Erlang version 17.1

[Michael Klishin]

Only mirrored queues will have a new master elected (there are certain scenarios

where this may or may not be the case, described in http://www.rabbitmq.com/ha.html).

There will be log entries about queue master movements.

While at it, consider not using 3.3.x for any tests. I don't see any reason to use it over 3.6.5 or at least 3.5.7.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Mark Wick]

Thanks for the reply.

We are using mirrored queues but it doesn't seem like there is a new master elected.  What should I be looking for to make sure this is happening?

Also, we will be looking to upgrade to new versions in the future but 3.3.1 is what I've got to work with now :-(

Thanks

On Thursday, October 13, 2016 at 3:42:55 PM UTC-6, Michael Klishin wrote:

Only mirrored queues will have a new master elected (there are certain scenarios

where this may or may not be the case, described in http://www.rabbitmq.com/ha.html).

There will be log entries about queue master movements.

While at it, consider not using 3.3.x for any tests. I don't see any reason to use it over 3.6.5 or at least 3.5.7.

On Thu, Oct 13, 2016 at 11:13 PM, Mark Wick <mmmm...@gmail.com> wrote:

I have a 2 node cluster with HA queue mirroring.

I use "rabbitmqctl report" on the "slave" and see a bunch of queues for the vhosts we have defined

I stop the master

I run the "rabbitmqctl report" again and see nothing, always

Shouldn't I be seeing the mirror'd queues on the slave?  And shouldn't those now become anchored on the slave whereby the slave is now the master?

I guess I'm just a bit confused what may be going on here.

NOTE:  Rabbit version is 3.3.1 with Erlang version 17.1

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

Log files: do node observe each other as down in your test? (it takes a while: http://www.rabbitmq.com/nettick.html)

Are the mirrors in sync so they can be promoted? And so on.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Mark Wick]

I see this in my mirrored node in the log:

=INFO REPORT==== 14-Oct-2016::16:39:48 ===

rabbit on node 'rab...@rabbit-01-internal.cos-int1.copper-alpha.com' down

=INFO REPORT==== 14-Oct-2016::16:39:49 ===

node 'rab...@rabbit-01-internal.cos-int1.copper-alpha.com' down: connection_closed

In my master queue node, I just see this after stopping rabbitmq:

=ERROR REPORT==== 14-Oct-2016::16:39:48 ===

AMQP connection <0.741.0> (running), channel 0 - error:

{amqp_error,connection_forced,

            "broker forced connection closure with reason 'shutdown'",none}

=INFO REPORT==== 14-Oct-2016::16:39:48 ===

Halting Erlang VM

I stop the service in order to test node going away.

I've waited over 10 minutes with no additional logs on mirrored node indicating that he's taking over as master for the queue

I've got to be missing something here but I don't know what.

Thanks

[Mark Wick]

Is this what I should see when a queue is being taken over by a slave?

=INFO REPORT==== 14-Oct-2016::16:39:48 ===

Mirrored queue 'amq.gen-f5hMVSBUS4GICe9-DAQJWQ' in vhost '/sensu': Master <rab...@rabbit-02-internal.cos-int1.copper-alpha.com.1.19770.38> saw deaths of mirrors <rab...@rabbit-01-internal.cos-int1.copper-alpha.com.2.255.0>

On Friday, October 14, 2016 at 10:58:50 AM UTC-6, Mark Wick wrote:

I see this in my mirrored node in the log:

=INFO REPORT==== 14-Oct-2016::16:39:48 ===

rabbit on node 'rabbit@rabbit-01-internal.cos-int1.copper-alpha.com' down

=INFO REPORT==== 14-Oct-2016::16:39:49 ===

node 'rabbit@rabbit-01-internal.cos-int1.copper-alpha.com' down: connection_closed

[Michael Klishin]

This only means that a mirror went down and master noticed.