Split Brain Recovery

[rishav.s...@moonshinetechnology.com]

Hi,

While going through Hazelcast latest documentation, i came across this statement under Split Brain Recovery section : "If they have an equal number of members, then a hashing algorithm determines the merging cluster".

I didn't understand this. How exactly the hashing algo works here ? Can someone explain the internal details of this hashing algo ?

[Vassilis Bekiaris]

Hi,

when recovering from network partition, there are a number of criteria based on which a subcluster determines whether it should merge towards the other or not. When subclusters have the same number of data members, the exact algorithm is not really important; all that matters is to have a deterministic algorithm that will be executed on both sub-clusters and both will reach the same decision in terms of which subcluster should merge towards the other.

That said, if you want to follow the actual code that makes the merge decision, it is here for master branch: https://github.com/hazelcast/hazelcast/blob/ed39fff8b34128add775e63979b72b2423cbb52c/hazelcast/src/main/java/com/hazelcast/internal/cluster/impl/ClusterJoinManager.java#L767-L823

Cheers,

Vassilis

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/6e06347d-62f5-4f20-9451-d9cefa635aa3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rishav.s...@moonshinetechnology.com]

Thanks a lot for the info :)

[rishav.s...@moonshinetechnology.com]

How does Hazelcast detect N/w split in the cluster ? I suppose there would be some background task to do that. What is the name of that background task ?

Also, once there is a n/w split how would hazelcast start the merging process? Does the above same background task initiates the merge as well? 

[rishav.s...@moonshinetechnology.com]

Any thoughts on this?

On Tuesday, May 14, 2019 at 6:52:32 PM UTC+5:30, Rishav Shandilya wrote:

[Ali Gurbuz]

Is it possible that you've missed the Vassilis's reply below ?

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/hazelcast/FKoMT9kECtc/sVdHaHryAwAJ

On Tue, May 14, 2019 at 4:36 PM Vassilis Bekiaris <vbek...@gmail.com> wrote:

Hi,

when recovering from network partition, there are a number of criteria based on which a subcluster determines whether it should merge towards the other or not. When subclusters have the same number of data members, the exact algorithm is not really important; all that matters is to have a deterministic algorithm that will be executed on both sub-clusters and both will reach the same decision in terms of which subcluster should merge towards the other.

That said, if you want to follow the actual code that makes the merge decision, it is here for master branch: https://github.com/hazelcast/hazelcast/blob/ed39fff8b34128add775e63979b72b2423cbb52c/hazelcast/src/main/java/com/hazelcast/internal/cluster/impl/ClusterJoinManager.java#L767-L823

Cheers,

Vassilis

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/fd86e5d7-32e4-485d-ba34-de16b1a798d3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Ali Gurbuz
Distinguished Engineer
Mahir İz Cad. No:35, Altunizade, İstanbul
a...@hazelcast.com 
+90 507 857 7815
skype: isbiroglu
@aligurbuz

[rishav.s...@moonshinetechnology.com]

I have seen his comment. His answer is related to my question which i had asked earlier. But this time my question is different. My doubt is :-

Let's say i have deployed a cluster and its running. Now, once there is a n/w partitioning, how is Hazelcast going to detect that? I suppose there is some background job which constantly monitors the cluster for split brain. So that background job will detect it. Once split brain has been detected, is it the same background task which initiates the merge process as well? If yes, what's the name of it? 

On Tuesday, May 14, 2019 at 6:52:32 PM UTC+5:30, Rishav Shandilya wrote:

[Ozan Kılıç]

Master node periodically checks if there is a separate cluster. 

If found, it sends SplitBrainJoinMessage to the other cluster and the merge process starts. 

Here you can see the background job scheduled: https://github.com/hazelcast/hazelcast/blob/7bd5433ea4ee1c831bc8bfce4ce127f92bb12684/hazelcast/src/main/java/com/hazelcast/internal/cluster/impl/ClusterServiceImpl.java#L159

On Fri, May 31, 2019 at 1:40 PM <rishav.s...@moonshinetechnology.com> wrote:

How does Hazelcast detect N/w split in the cluster ? I suppose there would be some background task to do that. What is the name of that background task ?

Also, once there is a n/w split how would hazelcast start the merging process? Does the above same background task initiates the merge as well? 

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/ce3ac994-bb40-4bde-8dd3-d26b2b13fb07%40googlegroups.com.

[rishav.s...@moonshinetechnology.com]

Thanks for the info.

On Tuesday, May 14, 2019 at 6:52:32 PM UTC+5:30, Rishav Shandilya wrote: