Hazelcast web session data expiration or expiry if pretty non-intuitive and poorly documented

[jon.st...@quantumsi.com]

Hello,

I am having lot of issues getting data expiration or timeouts working in tomcat 7 usining hazelcast clients to a stand alone hazelcast web cluster. 

Essentially I would like to make a hazelcast-wm accessing a hazelcast instance via a client in tomcat BUT I want my sessions (and all of the session data to expire say every 30 minutes if the session is inactive).

The ONLY way I seem to be able to evict anything is to set <time-to-live-seconds> or <max-idle-seconds> on the map name "my-sessions" nothing else works for me. I would love to use session-ttl-seconds, <session-timeout> or session.setMaxInactiveInterval(60*30) in my servlet code.

Under the servlet spec I should be able to poke attributes and get attributes as much as I want in a session and have everything stick around until the session expires via a sequence like

      HttpSession session = request.getSession();

      session.setAttribute(attName1, attValue1);

      session.setAttribute(attName2, attValue2);

      session.setAttribute(attName3, attValue3);

Then have all my session data "stick around" as long as I access either the session (but doing the below all my data is evicted):

      HttpSession session = request.getSession();

or access a session and one attribute (but doing the below attName1 and attName3 are evicted):

      HttpSession session = request.getSession();

      String s = (String) session.setAttribute(attName2, attValue2);

Any advice or insight would be appreciated (links to my configuration files are at the bottom) I am not sure if I have I have some sort of syntax issue or there is a bug in the 3.2.1 release of hazelcast.  Yet I'm so frustrated I think I am ready to fork the hazelcast-wm and expose some methods in WebFilter.java to work more directly with hazelcast to do what I want WRT expiry.

First. these methods (above) all fail to "expire" hazelcast 3.2.1 sessions

The below should work but it doesn't it only impacts the JSESSIONID (not the hazelcast session)

HttpSession session = request.getSession();

session.setMaxInactiveInterval(0); // never removed

session.setMaxInactiveInterval(60*60*24); // keep for 1 day

session.setMaxInactiveInterval(60*60); // keep for 1 hour

session.setMaxInactiveInterval(60*5); // keep for 5 min

 

I am having lot of issues getting data expiration or timeouts working in hazelcast tomcat & via hazelcast web clustering. The below should work but it doesn't it only impacts the JSESSIONID (not the hazelcast session).

I have read (via google searches) that the std web.xml <session-config><session-timeout>30</session-timeout></session-config> doesn't work. I read this LINK:

Thanks, so I have to set the TTL in the hazelcast config? The WebFilter doesn't respect the web.xml setting?

A: YES

So I thought that this should work: <init-param><param-name>session-ttl-seconds</param-name><param-value>1800</param-value></init-param> looking in the Hazelcast GIT source I do see the following in the file com/hazelcast/web/WebFilter.java

String sessionTTL = getParam("session-ttl-seconds"); 

Yet when putting session-ttl-seconds in either web.xml or even the hazelcast.xml (under the map name or not under the map name) it just doesn't work (I am runnign a client should it go in the hazelcast-client.xml ?)

 

So there seems only one way (actually two different settings) to get hazelcast web clusting sesssions to "sort of" expire - I show this below.

Notes 1) I didn't alter the default map name which is "my-sessions" I though about nameing it JSESSIONID but didn't 2) I do not add the <distributable /> tag to the web.xml.

The below settings in the hazelcast.xml "sort of" work to "expire" sessions and data but it doesn't work as expected

Given the map-name which is the name of my-sessions for the distributed map storing your web session objects located in <servlet_name>/MyServletExamplesClient/WEB-INF/web.xml and installed we see the syntax:

  <filter>
    <filter-name>hazelcast-filter</filter-name>
    <filter-class>com.hazelcast.web.WebFilter</filter-class>
        <init-param>
            <param-name>map-name</param-name>
            <param-value>my-sessions</param-value>
        </init-param>
        *
        *

Then by matching the "map name" hazelcast.xml which controls the hazelcast instance (we attache via hazelcast clients ftom tomcat 7 servlets) we can control "almost" the eviction of web sessions. A value of 1800 for max-idle-seconds. Note our distributed hazelcast persistent servlet session stuff is under hazelcast.sessionId (and has nothing to do with JSESSIONID).

    <map name="my-sessions">
        <in-memory-format>BINARY</in-memory-format>
        <backup-count>1</backup-count>
        <async-backup-count>0</async-backup-count>

        <!-- WORKS: if max-idle-seconds > 0 always removed even if the web session is not idle -->

        <time-to-live-seconds>0</time-to-live-seconds>

        <!-- WORKS: if max-idle-seconds > 0 only get's removed if the web 

             session goes idle, HOWEVER there seems to be a caveat "idle" 

             in this case means that a key-value pair is not accessed. When 

             I test doing a HttpSession session = request.getSession(); and 

             just getting the session without any session data for some 

             reason does not keep the session and it's data non-idle -->

        <max-idle-seconds>1800</max-idle-seconds>

        <!-- FAILS: if session-ttl-seconds > 0 only get's removed if the web session goes idle -->
        <!-- <session-ttl-seconds>30</session-ttl-seconds> -->

        <eviction-policy>NONE</eviction-policy>
        <max-size policy=\"PER_NODE\">0</max-size>
        <eviction-percentage>25</eviction-percentage>
        <merge-policy>com.hazelcast.map.merge.PassThroughMergePolicy</merge-policy>
    </map>

Setting max-idle to 1800 in a 7 node cluster and accessing the session every 10 minutes as follows (via a load balancer):

HttpSession session = request.getSession(); 

cd = new Date(session.getCreationTime());

la = Date(session.getLastAccessedTime();

 resulted in 21 different sessions being created, (about 40 minutes apart - suspicious timing here) this is not what I expected 

repeating the above but accessing only one of three (3) keys only the accessed key persists.

Let's look at the problem if only the sessionid is accessed and the session data isn't touched the session disappears!

Now if I repeat the test and access ALL of the KEY VALUE pairs in a session everything works - it seems like the <max-idle-seconds>1800</max-idle-seconds> setting is a per item thing but only for KEY VALUE session data.  What if I set three keys in a session and access one key one a minute for three hours and then at the end of three hours I try to find my other two keys - I would think that the session expiration would be "all or nothing".  

This all might make sense if hazelcast is storing items as follows: 

key = <hazelcast.sessionId>+<dataname>

value = datavalue

store (key, value) expire in 30  minutes

Yes it seems that way from the 3.2.1 hazelcast source

private String buildAttributeName(String name) {

return id + HAZELCAST_SESSION_ATTRIBUTE_SEPARATOR + name;

}

 

In this case for a work around, I could just encode a single JSON structure (and get everything at once even if I only want one field for a given access) and all would work as I would like.   But it just seems wrong .... as I said under the servlet spec I should be able to poke attributes and get attributes as much as I want in a session and have everything stick around until the session expires.

I have the same behavior with a simple one hazeclast instance and one non-load blaanced tomcat web server here are my configuration files

my web.xml ( http://pastie.org/9146298 )   look for <param-name>FAILSsession-ttl-seconds</param-name> I added FAILS

my hazelcast.xml  ( http://pastie.org/9146304 )  look for <max-idle-seconds>1800</max-idle-seconds> 

my hazelcast-client.xml ( http://pastie.org/9146306 ) 

I see various questions on the web seems like there are lots of issues that bite people (not just me).

http://stackoverflow.com/questions/22411544/hazelcast-map-expiration

http://stackoverflow.com/questions/19274887/hazelcast-map-ttl-not-working-with-spring-cache-abstraction

https://github.com/hazelcast/hazelcast/issues/955

Thanks in Advance

Jon