hazelcast replicated map replication problem

[gbal...@gmail.com]

Hi,

I used replicatedMap data structure in my stand-alone Java application. 

I got one problem about this. 

I have 5 hazelcast nodes on seperate machines. (all machines have 4 nodes)

Last week, this machines' connections was too slow.(normally ping time is 7 ms, last week it was about 50 ms )

And, because of this problem, replication was also too slow. It is normal.

 But, this replication's slowness affected my stand alone Java application performance. After the map.put() call, I do some another processes and they waited for replication delay.

I think the replication process is done another thread in the background, but likely it is not. 

I read the documentation of Hazelcast, it talks about replication delay millis. I set this value to 0, can my problem be based on this value? 

If it is set bigger than 0, is the replication done in background?

Thanks.

ReplicatedMapConfig replicatedMapConfig = new ReplicatedMapConfig();

replicatedMapConfig.setName("abcMap");

replicatedMapConfig.setReplicationDelayMillis(0);

config.addReplicatedMapConfig(replicatedMapConfig);

HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);

ReplicatedMap<String, String> hazelcastMap = hazelcastInstance.getReplicatedMap("abcMap");

[Enes Akar]

Replication is always async. But when you have a positive ReplicationDelayMillis; updates are sent as batches. So this is good for performance. 

When you set it to 0; updates will be sent one by one, that can decrease performance. 

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/9753aa98-ad09-404b-8553-57229857cbca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gbal...@gmail.com]

Thanks for your reply.

I simulated the slow connection state on my local machine and test machine at same network.

I used NetBalancer application to realize this state.

When I limited the network and did some delayes, the put process to the replicated map was about 1-2 second.

If I disabled the delay process on network, the put process is about 3-4 millisecond.

I think, the async replication of replicated map may have some problems. I cannot solve this problem.

I wait for your information.

Thanks.

24 Ocak 2017 Salı 15:54:40 UTC+3 tarihinde Enes Akar yazdı:

[diveplan...@gmail.com]

We're seeing the same thing on Hazelcast 3.10.2.

I.e. if we use a ReplicatedMap with two instances, which should be replicating asynchronously, updates are happening around 100 times slow (from the calling codes point of view) than when done with just a single local instance.

If I've understood correctly, with the asynchronous replication of a ReplicatedMap, shouldn't the delay caused by the network be invisible to the code doing "puts" into the map?

Regards,

-Nathan

[Sertuğ Kaya]

Replicated Map configs might be different in 3.10 as you can see from the link. Can you share your configs?

[nat...@upsideenergy.co.uk]

We've been tweaking other things, but coming back to our Replicated Map issue, we haven't done any specific config for either replicated maps in general, or this replicated map in particular.

We're using three embedded hazelcast instances, one in each of three running instances of our application.

We're using a Hazelcast portable serializer.

I realise that when we have three hazelcast instances in the cluster, we're going to incur the serialisation cost. However we're seeing a slow down of around 50ms, which seems high for serialisation.

Have we understood correctly that the replication in a ReplicatedMap, even in 3.10, is asynchronous?

[Sertuğ Kaya]

Yes, also please see the related section in the docs for other caveats about ReplicatedMap. https://docs.hazelcast.org//docs/3.10.3/manual/html-single/index.html#replicated-map

Feel free to share logs to see if there's any other problem in action that causes latency.