Tomcat dropping sessions ...

[DALDEI]

I run a cluster with 2 or 3 tomcat hosts on AWS using msm 1.9.3 and the AWS ElasticCache module (replacing spymemcached).

I use a Load Balancer with cookie session stickiness on JSESSIONID

Most of the time this works flawlessly but once in a while I get a state where sessions just drop ... 

The user experience is every click brings them back to the login page (because the session doesnt have their authentication)

Logging in simply brings them back.   The only way I have found to fix this is to kill the bad instance and then the rest work fine

and once the AutoScaling group replaces the bad instance all is well for days or weeks.

Before I start digging deeper into this I am wondering if anyone offhand has seen a similar problem or have any ideas where to start debugging.

Configuration looks like this:

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

    memcachedNodes="odd.lkqfb2.cfg.use1.cache.amazonaws.com:11211"

    requestUriIgnorePattern=".*/ncom_.*|.*/scratch|.*/heartbeat|.*/odd/|.*\.(ico|png|gif|jpg|css|js)$"

    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"

        sessionBackupAsync="true"

        sticky="true"

        copyCollectionsForSerialization="false"

    />

-David

[cosjav]

I'm experiencing the same kind of problem with a similar setup.

We are using Apache as the load balancer with sticky sessions on JSESSIONID and balanced across 2 tomcats.

On occasion tomcat re-issues a session when the previous one was perfectly fine and this seems to almost always be when images are requested but its not reliably reproducible.

I'm really stuck on this one and would really appreciate if anyone can help.

Thanks.

[Tristan Emerson]

Just adding another voice here, experiencing the same issue.  We're using Apache to load balance with sticky sessions on JSESSIONID, and running three Tomcat 7 instances.  Every once in a while a user's session will be lost and they will be redirected to the login page on their next request.  There's nothing in the Apache or Tomcat logs to help.

Apache 2.4.9 on Windows Server 2008 R2

Tomcat 7.0.47 on CentOS 6.5

MSM 1.8.0

Relevant section of context.xml is:

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

                memcachedNodes="n1:172.16.174.83:11211,n2:172.16.174.84:11211,n3:172.16.174.85:11211"

                failoverNodes="n1"

                requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"

        /> 

[cosjav]

I forgot to mention that we are using AJP between Apache and the Tomcat instances. Also when we tried going direct to Tomcat by bypassing Apache we weren't able to reproduce the problem. 

However, unfortunately (and fortunately at the same time) the problem suddenly disappeared (without changes) and has been unreproducible for the last 3 weeks - so we can't exactly point the finger at Apache or even further diagnose the problem.

It would be great (for my sanity) to get to the bottom of this...

[Martin Grotzke]

Just to let you know: I'm not using msm on AWS so I don't think I can do
more than you can do to analyse this issue. Of course I can answer
specific questions.

Cheers,
Martin

> <http://odd.lkqfb2.cfg.use1.cache.amazonaws.com:11211>"

>
> requestUriIgnorePattern=".*/ncom_.*|.*/scratch|.*/heartbeat|.*/odd/|.*\.(ico|png|gif|jpg|css|js)$"
>
> transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
> sessionBackupAsync="true"
> sticky="true"
> copyCollectionsForSerialization="false"
> />
>
>
> -David
>

> --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "memcached-session-manager" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to memcached-session-...@googlegroups.com
> <mailto:memcached-session-...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
inoio gmbh - http://inoio.de
Schulterblatt 36, 20357 Hamburg
Amtsgericht Hamburg, HRB 123031
Geschäftsführer: Dennis Brakhane, Martin Grotzke, Ole Langbehn

[Tristan Emerson]

I'm not using AWS either, but I am using AJP.  I'm actively working on finding the simplest example to reproduce this and will update as I find out more.  At this point I'm leaning toward this being an Apache->Tomcat issue, not a MSM issue.

On Tuesday, September 3, 2013 11:58:24 AM UTC-4, DALDEI wrote:

[Tristan Emerson]

Well as far as I can tell this has nothing to do with MSM.  I switched from AJP to HTTP and we haven't seen the problem since, but this is a very low load season for us. I'll give it another few days and update here if the problem comes back, but at this point I think it's a problem with the Apache AJP connector. 

[David Lee]

I have strong suspicions (and some but not conclusive) evidence that using JSESSSIONID with AWS ELB's doesn't work right.

When I changed from using JSESSIONID to using instead the ELB's own session cookie my problem went away.

Since then I have dropped using session caching (I am reworking to code to use my own kind of session instead of tomcats).

One possibility is that Tomcat can change the JSESSIONID from request to request even in the same session ...

to avoid some kind of security issue.  This may be whats confusing the ELB (and maybe AJP too).

 

I never tracked the full details but here's a hint

http://www.developer.com/java/web/article.php/3904871/Top-7-Features-in-Tomcat-7-The-New-and-the-Improved.htm

 

In any case - whether its tomcat or something else using session stickiness with the ELB's session cookie worked reliably,

where using JSESSION ID did not.

--

---
You received this message because you are subscribed to a topic in the Google Groups "memcached-session-manager" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/memcached-session-manager/g4SJb_G6LGA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to memcached-session-...@googlegroups.com.