Issues with getting logged out on AWS setup.

[Scott Wood]

I'm having some problems using the session stored in MSM, and I have a suspicion it may be related to my AWS setup. My problem is that, although the session is being saved in MSM, it doesn't appear to be used - I still get logged out and have to log back in when I turn off a Tomcat.

I'm using an ELB (sticky sessions on, app-stickiness) in front of two Tomcats running on EC2 instances, and have configured MSM to use Elasticache according to the wiki. I can telnet to Elasticache from both Tomcats and getting the key shows me the serialized data from the app I expect.

I tried some debugging, and when I Inspect the cookie with Developer Tools in Chrome, it's always called 'AWSELB'. The wiki suggested 'JSESSIONID', which is what I named the cookie when prompted on the ELB interface, but Amazon doesn't seem to support actually changing the name visible to the browser. Again, I am not entirely sure if this is why I'm experiencing the behaviour I am, it is just a hypothesis.

Does anyone have any experience using a setup like this, and can confirm if this is the problem? If so, how do I fix it? Please let me know if any other logging would be helpful here.

[Martin Grotzke]

I don't have experience with this setup, but you could also try to reconfigure tomcat to use AWSELB as session cookie name. I've never actually tested this for msm, so no guarantees :-)

Cheers,
Martin

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luis Arias]

Hi Scott,

The AWSELB cookie is not the same thing as the session cookie, it's just there to enforce session stickiness by the ELB as far as I understand.  Setting up the ELB to use the JSESSIONID cookie for sticky-sessions and app-stickiness is how it works for us.  I suspect another issue, likely related to your memcached session manager config on Tomcat.

Luis

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

[Scott Wood]

Hi Luis,

Thanks for your reply. For what it's worth, here's my context.xml manager element that is present on both Tomcats:

   <Manager

    className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

    memcachedNodes="test.uishek.cfg.use1.cache.amazonaws.com:11211"

    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"

    sessionBackupAsync="true"

    sticky="true"

    copyCollectionsForSerialization="false"

    />

I dug a little bit deeper after making my post, and trying both sticky and non-sticky scenarios, and seeing the same behaviour. The network logs show a large number of successful requests, but on shutdown, it appears my app is showing a couple of isolated 401 Unauthorized errors. There are a few successful (200) responses, then a bunch of canceled requests before logout. Barring any configuration option that might cause this, I think the problem could be within my app. I'll check into it today and report more findings.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Scott Wood]

After an intensive review, I realized the root of the problem was with my application, and it is sufficiently interesting that I thought it prudent to share.

A certain class was being stored in the session, and while it was serializable, its had fields which were not. Thus, it served as an effective "trojan horse" that snuck data into the session that couldn't be serialized. When MSM tried to unserialize data that couldn't be serialized in the first place, it threw a fit and aborted the process, deleting the session. Turning up the debug logs helped me to find the problem and address it by making the fields transient explicitly.

Now it works perfectly as expected! :)

One final thing I noticed during my testing was that, with application stickiness enabled, after stopping a Tomcat and switching over to another one, when the original primary is brought back up, the application will always switch back over to that original Tomcat. Is the reason for this related to how the application stickiness works?

[Martin Grotzke]

Hi Scott,

thanks a lot for sharing your findings! Do you think we can improve anything that would make it easier to detect the issue you encountered?

Regarding your question I'd say that this is not expected - the session id should have been changed and therefore your loadbalancer should continue to send the request to the second tomcat. To analyze this you should check the loadbalancer settings, tomcats jvmRoute and how the session cookie evolves.

Cheers,
Martin

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

[Scott Wood]

Hi Martin,

No problem at all! I don't think so - the issue is totally on my end and a result of previous session work/session handling having been done very poorly. The class being stored in question was actually a session manager itself (A session manager stored in a session - figure that one out!), and so some very naughty and hacked-together things were happening that I have since cleaned up. The logs were all I needed to diagnose the issue properly once I started debugging.

Thanks for the tips, I'll take a closer look at all of those and post any findings that might help others encountering similar behaviour.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

[Martin Grotzke]

Ok :-)

Cheers,
Martin

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

[Scott Wood]

I did find this reference which seems to indicate the opposite, so my curiosity is piqued as to what could be going on:

Application-Controlled Session Stickiness

The load balancer uses a special cookie to associate the session with the instance that handled the initial request, but follows the lifetime of the application cookie specified in the policy configuration. The load balancer only inserts a new stickiness cookie if the application response includes a new application cookie. The load balancer stickiness cookie does not update with each request. If the application cookie is explicitly removed or expires, the session stops being sticky until a new application cookie is issued.

If an instance fails or becomes unhealthy, the load balancer stops routing requests to that instance, and chooses a new healthy instance based on the existing load balancing algorithm. The load balancer treats the session as now "stuck" to the new healthy instance, and continues routing requests to that instance even if the failed instance comes back.

Ok :-)

Cheers,
Martin

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-manager+unsub...@googlegroups.com.

[Martin Grotzke]

I'd turn debug logs on, see what they tell and what the loadbalancer returns and try to make sense of it.

Cheers,

Martin 

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

"Think like a fundamentalist but code like a hacker" - @headinthebox

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "memcached-session-manager" group.

To unsubscribe from this group and stop receiving emails from it, send an email to memcached-session-...@googlegroups.com.

[Scott Wood]

I switched to Load Balancer managed stickiness and the problem seems to have disappeared. I'll chalk it up to mea culpa again and see if I can figure out what more ungodly nonsense the app is up to. Not to worry - it's not an MSM problem :) For now, using LB stickiness works great!

[Martin Grotzke]

Good to hear it's working now - or at least this issue is resolved :-)

--