Cluster join issue

[jasmeet kaur]

I have a existing cluster of 3 nodes and i have to join 3 new nodes to the existing cluster.

The task is being done by CFT so all the new nodes spawns and tries to join the existing cluster parallelly.

Now one of the node successfully joins the cluster but other two fails with this error:

 ** FATAL ** Failed to merge schema: {aborted,

                                      {combine_error,schema,

                                       [<NODE_Details>]}}

=ERROR REPORT==== 19-Nov-2018::02:57:11 ===

** Generic server mnesia_monitor terminating 

** Last message in was {'EXIT',<0.1136.0>,killed}

** When Server state == {state,<0.1136.0>,[],[],true,[],undefined,[],[]}

** Reason for termination == 

** killed

=ERROR REPORT==== 19-Nov-2018::02:57:11 ===

** Generic server mnesia_recover terminating 

** Last message in was {'EXIT',<0.1136.0>,killed}

** When Server state == {state,<0.1136.0>,undefined,undefined,undefined,0,

                               false,true,[]}

** Reason for termination == 

** killed

=ERROR REPORT==== 19-Nov-2018::02:57:11 ===

** Generic server mnesia_snmp_sup terminating 

** Last message in was {'EXIT',<0.1136.0>,killed}

** When Server state == {state,

                            {local,mnesia_snmp_sup},

                            simple_one_for_one,

                            [{child,undefined,mnesia_snmp_sup,

                                 {mnesia_snmp_hook,start,[]},

                                 transient,3000,worker,

                                 [mnesia_snmp_sup,mnesia_snmp_hook,

                                  supervisor]}],

                            undefined,0,86400000,[],0,mnesia_snmp_sup,[]}

** Reason for termination == 

** killed

=ERROR REPORT==== 19-Nov-2018::02:57:11 ===

** Generic server mnesia_subscr terminating 

** Last message in was {'EXIT',<0.1136.0>,killed}

** When Server state == {state,<0.1136.0>,962580}

** Reason for termination == 

** killed

=INFO REPORT==== 19-Nov-2018::02:57:11 ===

Error description:

   {could_not_start,mnesia,

       {{shutdown,{failed_to_start_child,mnesia_kernel_sup,killed}},

        {mnesia_sup,start,[normal,[]]}}}

[Michael Klishin]

You cannot join existing clusters as they have different schemas and internal

schema data store identifiers.

You can break a cluster down and add reset nodes one by one or use federation to connect them.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[jasmeet kaur]

But if I try to join the existing cluster separately all the nodes successfully join the cluster.

And for all the queues HA policy is enabled so they will be mirror and new nodes I am trying to add have same version of both rabbit mq and erlang and erlang cookie of new node is also same as existing node.

Can you elaborate what's the problem in this scenario?

[Michael Klishin]

Having the same version is not enough. A node with an initialised schema database cannot just join an arbitrary cluster.

This is why in the tutorial [1] the node is reset first and why peer discovery only works with blank (not yet initialised) nodes.

Mirroring is entirely orthogonal to cluster formation.

1. http://www.rabbitmq.com/clustering.html

[jasmeet kaur]

Yes I understand that the nodes with different schema cannot be made part of same cluster.

But the new node which I am try to add to a cluster does not have any data or schema because it's a fresh node.

Still I can't add to the cluster?

[Michael Klishin]

A node must be *reset* before it can be added to the cluster.

If you boot a node once it will form a cluster of one and won't be able to join any peers [1].

[2] covers this question in a different setting, please take a look.

1. http://www.rabbitmq.com/cluster-formation.html#peer-discovery-how-does-it-work

2. http://www.rabbitmq.com/cluster-formation.html#rejoining