Failover in hazelcast-tomcat web session replication

[sharad keer]

HI All,

We are using hazelcast-3.7.5 and planning to use its web session replication feature to manage user session in distributed cache. We are using hazelcast as Client/Server mode with tomcat-7.0.42 and  in sticky session mode. For load balancing we are using apache web server in local which works fine in case of any of the tomcat instance failure. 

In our one of the test scenario when we brought down hazelcast cluster and try to access the application, it is keep processing that request and after some time its stop working . By looking logs we have found following exception:-

 com.hazelcast.client.HazelcastClientNotActiveException: Partition does not have owner. partitionId : 94

        at com.hazelcast.client.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:170)

        at com.hazelcast.client.spi.impl.ClientInvocation.invoke(ClientInvocation.java:127)

        at com.hazelcast.client.spi.ClientProxy.invokeOnPartition(ClientProxy.java:152)

        at com.hazelcast.client.spi.ClientProxy.invoke(ClientProxy.java:147)

        at com.hazelcast.client.proxy.ClientMapProxy.removeInternal(ClientMapProxy.java:267)

        at com.hazelcast.client.proxy.NearCachedClientMapProxy.removeInternal(NearCachedClientMapProxy.java:138)

        at com.hazelcast.client.proxy.ClientMapProxy.remove(ClientMapProxy.java:261)

        at com.hazelcast.session.HazelcastSessionManager.remove(HazelcastSessionManager.java:326)

        at com.hazelcast.session.HazelcastSessionManager.remove(HazelcastSessionManager.java:292)

        at org.apache.catalina.session.StandardSession.expire(StandardSession.java:840)

        at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:658)

        at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:534)

        at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:519)

        at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1352)

        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1530)

        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1540)

        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1540)

        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1519)

I know this exception is coming because our client is disconnected from the hazelcast server. I wanted to know do we have any failure mechanism available in hazelcast for above scenario as there are number of scenario where whole hazelcast cluster will be  unreachable for any application, and in that case application will stop working and blocks all processing.

For reference below are the configuration details:-

Server.xml:-

<Listener className="com.hazelcast.session.ClientServerLifecycleListener" configLocation="hazelcast-client.xml"/>

hazelcast-client.xml

    <group>

<name>session-replication-cache</name>

<password>test@2002</password>

    </group>

    <network>

        <cluster-members>

            <address>127.0.0.1:5701</address>

</cluster-members>

        <smart-routing>true</smart-routing>

        <redo-operation>true</redo-operation>

<connection-attempt-period>5000</connection-attempt-period>

<connection-attempt-limit>60</connection-attempt-limit>

    </network>

<near-cache name="sessions">

<time-to-live-seconds>0</time-to-live-seconds>

<max-idle-seconds>0</max-idle-seconds>

<invalidate-on-change>true</invalidate-on-change>

<in-memory-format>BINARY</in-memory-format>

<local-update-policy>INVALIDATE</local-update-policy>

</near-cache>

We are using hazelcast-tomcat7-sessionmanager-1.1 and tomcat-catalina-7.0.42 integration for hazelcast and tomcat integration.For apache we server we also did few configuration in httpd.conf which is related to defining nodes and enabling the modules.

 

Quick help will be appreciated. Thanks in advance.  

Regards,

Sharad Keer

[em...@hazelcast.com]

Hi Sharad,

Currently we don't have any failover mechanism in our Tomcat Session Manager. I think you can temporarily solve it in a similar way to your comment on Hazelcast Hibernate plugin (https://github.com/hazelcast/hazelcast-hibernate/issues/32). Can you please create an issue for this? If you can contribute your solution to the Hazelcast Tomcat Session Manager plugin, that would be awesome.

Cheers,

Emre

[sharad keer]

Thanks Emre for your quick response. I'll follow the same and contribute this solution to the Hazelcast Tomcat Session Manager plugin.