Hazelcast cache issue
Alexey Musienko
Hi Neil,
We are using Hazelcast 3.8 for cache over the Spring Cache and time to time having problems.
We have the following landscape:
1) Hazelcast server. Look at cluster.xml in attachment.
2) Hazelcase client 1, which is running in the same JVM as Hazelcast server. Hazelcast client 1 works in Spring+Vert.x application.
Look at hazelcast-client_1.xml
3) Hazelcast client 2, which is running in another JVM to Hazelcast server. Hazelcast client 2 works in Spring Boot application.
Look at hazelcast-client_2.xml
4) Cache config for client 2 in spring boot:
import com.hazelcast.client.HazelcastClient;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.spring.cache.HazelcastCacheManager;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
@Configuration
@Profile("default")
@EnableCaching
public class CacheConfig {
@Bean(destroyMethod = "shutdown")
public HazelcastInstance hazelcastInstance() {
return HazelcastClient.newHazelcastClient();
}
@Bean
public CacheManager cacheManager(HazelcastInstance hazelcastInstance) {
return new HazelcastCacheManager(hazelcastInstance);
}
}
6) Hazelcast config for client 1:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:cache="http://www.springframework.org/schema/cache"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd">
<cache:annotation-driven cache-manager="cacheManager" />
<bean id="hazelcastClientInstance" class="com.hazelcast.client.HazelcastClient" factory-method="newHazelcastClient" destroy-method="shutdown">
</bean>
<bean id="cacheManager" class="com.hazelcast.spring.cache.HazelcastCacheManager">
<constructor-arg ref="hazelcastClientInstance"/>
</bean>
</beans>
7) We are using spring @CacheEvict, @Cacheable for cache management.
For example:
@CacheEvict(value = "merchantAccounts", allEntries = true) @Cacheable(value = "merchantAccounts")
@CacheEvict & @Cacheable are executed in different JVMs. The idea is the following: client 1 & client 2 see the distributed cache.
8) After using the platform few hours, we are getting the following an error:
11:36:31.650 [vert.x-worker-thread-61] [c990977b-4a01-4c68-a4a4-384d25323982] ERROR c.i.p.t.TransactionProcessorImpl - Error:
com.hazelcast.client.HazelcastClientNotActiveException: Partition does not have owner. partitionId : 155
at com.hazelcast.client.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:159) ~[transaction-service-4.42.0-fat.jar:na]
at com.hazelcast.client.spi.impl.ClientInvocation.invoke(ClientInvocation.java:119) ~[transaction-service-4.42.0-fat.jar:na]
at com.hazelcast.client.spi.ClientProxy.invokeOnPartition(ClientProxy.java:153) ~[transaction-service-4.42.0-fat.jar:na]
at com.hazelcast.client.spi.ClientProxy.invoke(ClientProxy.java:148) ~[transaction-service-4.42.0-fat.jar:na]
at com.hazelcast.client.proxy.ClientMapProxy.getInternal(ClientMapProxy.java:291) ~[transaction-service-4.42.0-fat.jar:na]
According to our log, we don't have any disconnections and an exception happens randomly.
