load balancing support?
ksucontract
of fail over to any arbitrary server in a cluster, and does not
provide support for load balancing requests to multiple tomcat
servers?
Also, it was not clear from the project description, but what exactly
happens during a failover? Assume we have a system with a cluster of 3
nodes node A, B, and C with 2 Apache load balancer A1 and A2 and a
hardware load balancer H in front of A1 and A2. If node A failed (or
is busy), and multiple requests came in with sessionId that points to
A as the destination, what happened if those requests were send to A1
and A2 in a round robin fashion, and A1 forward requests to B as the
fail-over and A2 forward requests to C as the fail-over?
huksley
failover (and load balancing) is handled by mod_jk.
If server which used in sticky session scenario is not available, this
session is distributed to another servers using current load balancing
scheme in use. Just like new incoming request without JSESSIONID.
Martin Grotzke
> I assume the design of memcached-session-manager is meant for the case
> of fail over to any arbitrary server in a cluster, and does not
> provide support for load balancing requests to multiple tomcat
> servers?
msm is made web applications with sticky-sessions:
you (or your loadbalancer) is responsible for assigning a user to a
certain tomcat, so that all requests of this user are going to this
tomcat.
This is independent of the number of layers that are taking part in
loadbalancing.
However, there's already an (experimental) branch for
nonsticky-sessions which provides absolutely request / the most basic
functionality required for nonsticky-sessions: sessions are loaded
from memcached if requested by the application for each request, when
the request is finished the session is removed from this tomcats
session map.
The challenge with nonsticky-sessions are concurrent requests, as a
concurrent modification of the session is just not handled and the
request finishing last wins (session is stored at the end of the
request in memcached).
As right now there's no single-point-of-failure as sessions are stored
both in tomcat and memcached (and the failoverNode setting so that
sessions can be stored on different machines / in different data
centers) with nonsticky-sessions (sessions are only stored permanently
in memcached) there would also need to be another strategy to prevent
a SPOF - probably sessions would be stored additionally in a secondary
memcached for backup.
>
> Also, it was not clear from the project description, but what exactly
> happens during a failover? Assume we have a system with a cluster of 3
> nodes node A, B, and C with 2 Apache load balancer A1 and A2 and a
> hardware load balancer H in front of A1 and A2. If node A failed (or
> is busy), and multiple requests came in with sessionId that points to
> A as the destination, what happened if those requests were send to A1
> and A2 in a round robin fashion, and A1 forward requests to B as the
> fail-over and A2 forward requests to C as the fail-over?
In the best case there would be no problem (when requests are read
only and no session migration is initiated).
In the normal case this situation would cause unpredictable behavior.
There could be data loss due to parallel writes to the same data and
there could also be a session migration initiated (e.g. session
migrated from memcached1 to memcached2 for a request served by a
tomcat living on the same machine as memcached1 with memched1
configured as failoverNode) which causes the change of the JSESSIONID
and an appropriate Set-Cookie header in the corresponding response.
You would need to configure the hardware loadbalancer to do sticky
sessions (so that those requests would be forwarded to only one
apache) and configure your apaches to do also sticky sessions (or
setup apache-tomcat pairs, so that each apache is fronting exactly one
tomcat).
However, I'm also interested in pushing the nonsticky-sessions branch.
What comes to my mind is some kind of locking, so that sessions are
locked by requests and other requests (on different tomcats) can
access the session as soon as the other request is finished. Perhaps
one would like to lock sessions only for POST/PUT/DELETE requests but
not for GET requests, but this should only be an optional setting for
users that really know which http (ajax) requests are performed.
Another task would be to solve the aforementioned SPOF issue, e.g. by
storing sessions in two memcached nodes (a primary and a secondary).
What do you think?
Cheers,
Martin
ksucontract
sticky session manager is what I am looking for. If you could push
the nonsticky-sessions branch of memcached-session-manager, I am
definitely interested in taking a look at it.
Best Regards,
Kevin
P.S. Terracotta Web Sessions is another interesting prospect I am
looking at.
On Dec 30, 10:22 pm, Martin Grotzke <martin.grot...@googlemail.com>
wrote:
Martin Grotzke
> A high performance, highly scalable, fault tolerant (i.e. no SFOF) non-
> sticky session manager is what I am looking for. If you could push
> the nonsticky-sessions branch of memcached-session-manager, I am
> definitely interested in taking a look at it.
https://github.com/magro/memcached-session-manager/tree/nonsticky-sessions
What do you say regarding the proposed solutions for parallel requests and SPOF?
What kind of parallel requests do you have btw.? I assume this are
also ajax requests. Is there a request pattern that allows to
distinguish between readonly and read/write requests (regarding
session access/modification)?
> P.S. Terracotta Web Sessions is another interesting prospect I am
> looking at.
We also considered this product but decided against it due to these reasons:
- We couldn't find detailed information on how the internals work
(also the technical docs said only s.th. regarding weaving etc., but
not that much about the process internals), it made the impression of
too much magic (too little insight)
- Closed source, so we would not have been able to change/implement
some urgent need (or inspect the source to see what's going on)
- Closed source + commercial product + comercial support: this would
have taken control away from us
- Additional infrastructure for terracotta that needed to make it's
way into the operations team (that did not have experience with this
product)
So we created msm to fullfill our need of high-performance,
scalability and transparency/control. It was created for tchibo (.de,
.at, .ch, .cz and other domains), one of the biggest ecommerce sites
btw.
However, since then I always wanted to make a detailed feature and
performance comparison of TC Web Sessions, tomcat session replication
and msm, but didn't find the time. So if you should plan to do s.th.
like this I'd definitely be interested (in helping if wanted).
Cheers,
Martin
PS: if you sign up as user on the google groups I don't have to
moderate/approve your posts - I activated moderation of messages of
new/unsubscribed users so that spam doensn't hit this list.
Martin Grotzke
I have yet another question: which web framework are you using? What
kind of data are you storing in the sesssion?
I ask because for some frameworks there are optimizations possible or
custom serializers required. E.g. wicket accesses thread locals from
components stored in the session, or hibernate persistent collections
access the session on instantiation/deserialization. JodaTime has a
rather big DateTime/Timezone which would be serialized by default but
it can be optimized significantly.
Cheers,
Martin
On Fri, Dec 31, 2010 at 7:11 AM, ksucontract <ksuco...@gmail.com> wrote:
--
Martin Grotzke
http://www.javakaffee.de/blog/
ksucontract
The type of data stored is (currently) typical of eComm, shopping cart
related data. Thanks for the tips on Wicket and JodaTime! We'll
certainly optimize the serialization / deserialization of data as
needed.
-Kevin
On Dec 31 2010, 9:08 pm, Martin Grotzke
ksucontract
On Dec 31 2010, 7:54 pm, Martin Grotzke
<martin.grot...@googlemail.com> wrote:
> > A high performance, highly scalable, fault tolerant (i.e. no SFOF) non-
> > sticky session manager is what I am looking for. If you could push
> > the nonsticky-sessions branch of memcached-session-manager, I am
> > definitely interested in taking a look at it.
>
> What do you say regarding the proposed solutions for parallel requests and SPOF?
> What kind of parallel requests do you have btw.? I assume this are
> also ajax requests. Is there a request pattern that allows to
> distinguish between readonly and read/write requests (regarding
> session access/modification)?
standard out of shelf memcached server? As most access will probably
be ready only... some sort of read / write lock will probably be
needed, and that's probably best done at the server. Locking based on
request type will probably be adding too much constraint to the
session manager - as POST / DELETE / PUT may not necessary translates
to a modify to the session object itself, but rather to the business
objects.
>
> > P.S. Terracotta Web Sessions is another interesting prospect I am
> > looking at.
>
> We also considered this product but decided against it due to these reasons:
> - We couldn't find detailed information on how the internals work
> (also the technical docs said only s.th. regarding weaving etc., but
> not that much about the process internals), it made the impression of
> too much magic (too little insight)
> - Closed source, so we would not have been able to change/implement
> some urgent need (or inspect the source to see what's going on)
> - Closed source + commercial product + comercial support: this would
> have taken control away from us
> - Additional infrastructure for terracotta that needed to make it's
> way into the operations team (that did not have experience with this
> product)
>
> So we created msm to fullfill our need of high-performance,
> scalability and transparency/control. It was created for tchibo (.de,
> .at, .ch, .cz and other domains), one of the biggest ecommerce sites
> btw.
>
> However, since then I always wanted to make a detailed feature and
> performance comparison of TC Web Sessions, tomcat session replication
> and msm, but didn't find the time. So if you should plan to do s.th.
> like this I'd definitely be interested (in helping if wanted).
the mgmt type to see if the results could be shared, and if a
collaborate study is possible. From my perspective, your help
certainly will be very valuable and appreciated.
>
> Cheers,
> Martin
Sorry for the late response. It is the holiday season :)
Martin Grotzke
> On Dec 31 2010, 7:54 pm, Martin Grotzke <martin.grot...@googlemail.com> wrote:
>> What do you say regarding the proposed solutions for parallel requests and SPOF?
>> What kind of parallel requests do you have btw.? I assume this are
>> also ajax requests. Is there a request pattern that allows to
>> distinguish between readonly and read/write requests (regarding
>> session access/modification)?
>
> My first question is how would one synchronized the locks with a
> standard out of shelf memcached server?
I created an issue for locking and already implement a simple
pessimistic locking strategy:
http://code.google.com/p/memcached-session-manager/issues/detail?id=71
https://github.com/magro/memcached-session-manager/commit/db8afac4
The locking is implemented like this:
http://code.google.com/p/memcached/wiki/FAQ#Emulating_locking_with_the_add_command
> As most access will probably
> be ready only...
I'd say this depends on the application.
> some sort of read / write lock will probably be
> needed, and that's probably best done at the server.
I don't find a r/w lock suitable here, either the session is locked
for modification or not. The r/w lock would add that a (write) lock
cannot be obtained as long as any read lock is open. We don't need
this.
Therefore the question is, how to determine the requests for that no
lock has to be aquired.
At the end of each request (for that a session exists) it's checked if
the session was modified.
In some sessionLockingMode="detect" msm could see if there are request
patterns that always don't modify the session and skip locking for
such requests.
It should also be possible to specify request patterns for that locks
are not aquired (or patterns for that locks are aquired), and I'd
prefer explicitly specified patterns over automagic detection (though
it would be nice to provide pattern detection).
It should also be possible to specialize this by subclassing the
MemcachedBackupSessionManager, so that very application specific logic
could be introduced.
> Locking based on
> request type will probably be adding too much constraint to the
> session manager - as POST / DELETE / PUT may not necessary translates
> to a modify to the session object itself, but rather to the business
> objects.
Yes, I agree.
Regarding the data stored in the session, may I ask if you're using /
going to use wicket, or which web-framework you chose?
Cheers,
Martin
Martin Grotzke
<martin....@googlemail.com> wrote:
> On Tue, Jan 4, 2011 at 2:23 AM, ksucontract <ksuco...@gmail.com> wrote:
>> As most access will probably
>> be ready only...
> I'd say this depends on the application.
Just to clarify: sessions are only locked for requests that _access_
the session. For requests that are just "running through" without any
request.getSession() call the session is (of course) not loaded from
memcached.
Cheers,
Martin
Kevin Su
On Tue, Jan 4, 2011 at 2:23 AM, ksucontract <ksuco...@gmail.com> wrote:
> On Dec 31 2010, 7:54 pm, Martin Grotzke <martin.grot...@googlemail.com> wrote:
>> What do you say regarding the proposed solutions for parallel requests and SPOF?I created an issue for locking and already implement a simple
>> What kind of parallel requests do you have btw.? I assume this are
>> also ajax requests. Is there a request pattern that allows to
>> distinguish between readonly and read/write requests (regarding
>> session access/modification)?
>
> My first question is how would one synchronized the locks with a
> standard out of shelf memcached server?
pessimistic locking strategy:
http://code.google.com/p/memcached-session-manager/issues/detail?id=71
https://github.com/magro/memcached-session-manager/commit/db8afac4
The locking is implemented like this:
http://code.google.com/p/memcached/wiki/FAQ#Emulating_locking_with_the_add_command
> As most access will probablyI'd say this depends on the application.
> be ready only...
I don't find a r/w lock suitable here, either the session is locked
> some sort of read / write lock will probably be
> needed, and that's probably best done at the server.
for modification or not. The r/w lock would add that a (write) lock
cannot be obtained as long as any read lock is open. We don't need
this.
Therefore the question is, how to determine the requests for that no
lock has to be aquired.
At the end of each request (for that a session exists) it's checked if
the session was modified.
In some sessionLockingMode="detect" msm could see if there are request
patterns that always don't modify the session and skip locking for
such requests.
It should also be possible to specify request patterns for that locks
are not aquired (or patterns for that locks are aquired), and I'd
prefer explicitly specified patterns over automagic detection (though
it would be nice to provide pattern detection).
It should also be possible to specialize this by subclassing the
MemcachedBackupSessionManager, so that very application specific logic
could be introduced.
Yes, I agree.
> Locking based on
> request type will probably be adding too much constraint to the
> session manager - as POST / DELETE / PUT may not necessary translates
> to a modify to the session object itself, but rather to the business
> objects.
Regarding the data stored in the session, may I ask if you're using /
going to use wicket, or which web-framework you chose?
Cheers,
Martin
Martin Grotzke
> On Tue, Jan 4, 2011 at 10:57 PM, Martin Grotzke <martin....@googlemail.com> wrote:
>> I created an issue for locking and already implement a simple
>> pessimistic locking strategy:
>> http://code.google.com/p/memcached-session-manager/issues/detail?id=71
>> https://github.com/magro/memcached-session-manager/commit/db8afac4
>>
>> The locking is implemented like this:
>>
>> http://code.google.com/p/memcached/wiki/FAQ#Emulating_locking_with_the_add_command
>>
>
> Nice. Didn't know memcached supports locking. I think there is a need to
> support locking around access / modify of session data from memcached.
> Something like:
>
> try {
> MSM.lock(sessionId);
> Session session = MSM.get(sessionId)
> .... modify session
> }
> finally {
> MSM.unlock(session);
> }
> Not perfect since application is exposed to the type of cache management
> solution. However, to guarantee no data is overridden, it seems necessary.
So what you want is to lock the session from your application? Sounds
reasonable, the advantage is minimal locking scope. I'll see if it's
possible to access msm from the application (different classloaders
hierarchy in tomcat, there are some subtle difficulties with this).
To guarantee that no data is overwritten you can also use the
pessimistic locking I already implemented (mentioned above), did you
see those links?
http://code.google.com/p/memcached-session-manager/issues/detail?id=71
https://github.com/magro/memcached-session-manager/commit/db8afac4
I'm currently working on the "automatic" locking mode that detects
session readonly requests and skips locking for them.
Another locking strategy I'd implement would be based on a
specified/configured pattern. IIRC I mentioned both already.
> By the way, are there any performance results for sticky-less msm yet?
Nope. I'm just working an the locking solutions, but afterwards it
would make sense to run a benchmark.
However, the results will vary depending on the hardware
(serialization/deserialization is CPU bound), network infrastructure
(tomcat <-> memcached communication) and application (number of
requests with/without session, number of requests changing / not
changing the session, the session data that's serialized etc.).
So I would do a benchmark to get a rough impression what are the
differences in one/two/three situations.
If you want to know for sure how your application will behave in your
specific infrastructure/setup you definitely should run performance
tests in your environment with your application.
>> > some sort of read / write lock will probably be
>> > needed, and that's probably best done at the server.
>> I don't find a r/w lock suitable here, either the session is locked
>> for modification or not. The r/w lock would add that a (write) lock
>> cannot be obtained as long as any read lock is open. We don't need
>> this.
>> Therefore the question is, how to determine the requests for that no
>> lock has to be aquired.
>>
>> At the end of each request (for that a session exists) it's checked if
>> the session was modified.
>> In some sessionLockingMode="detect" msm could see if there are request
>> patterns that always don't modify the session and skip locking for
>> such requests.
>> It should also be possible to specify request patterns for that locks
>> are not aquired (or patterns for that locks are aquired), and I'd
>> prefer explicitly specified patterns over automagic detection (though
>> it would be nice to provide pattern detection).
>
> For auto-detect mode, what if the the request did not modify the session
> because, at the time, the business logic decided no update was needed.
msm will detect this (comparing the serialized bytes of the session at
the start/end of the request) and skip sending the session to
memcached.
> However, session may get updated later if the data had changed? (e.g. if
> the user did something more than N times, update a flag in the session)
I'm currently using some kind of blacklist, so that a request is no
longer considered to be readonly as soon as one request of the same
pattern modified the session.
I already thought about a threshold for readonly-detection (so that
100 requests of a certain pattern must pass before this pattern is
regarded as readonly), which should also be configurable.
>> Regarding the data stored in the session, may I ask if you're using /
>> going to use wicket, or which web-framework you chose?
>
> There is no plan to use wicket. Some of the frameworks in our mix are
> Struts and Spring.
Ok, which versions? As both are "plain" MVC frameworks they shouldn't
do anything special with the session in terms of serialized data, so
the session structure depends completely on your application. So you
are in the great situation that you can keep your session very small
if you store only ids where possible.
Cheers,
Martin
Martin Grotzke
<martin....@googlemail.com> wrote:
> On Thu, Jan 6, 2011 at 5:04 PM, Kevin Su <ksuco...@gmail.com> wrote:
>> On Tue, Jan 4, 2011 at 10:57 PM, Martin Grotzke <martin....@googlemail.com> wrote:
>>> I created an issue for locking and already implement a simple
>>> pessimistic locking strategy:
>>> http://code.google.com/p/memcached-session-manager/issues/detail?id=71
>>> https://github.com/magro/memcached-session-manager/commit/db8afac4
>>>
>>> The locking is implemented like this:
>>>
>>> http://code.google.com/p/memcached/wiki/FAQ#Emulating_locking_with_the_add_command
>>>
>>
>> Nice. Didn't know memcached supports locking. I think there is a need to
>> support locking around access / modify of session data from memcached.
>> Something like:
>>
>> try {
>> MSM.lock(sessionId);
>> Session session = MSM.get(sessionId)
>> .... modify session
>> }
>> finally {
>> MSM.unlock(session);
>> }
>> Not perfect since application is exposed to the type of cache management
>> solution. However, to guarantee no data is overridden, it seems necessary.
>
> So what you want is to lock the session from your application? Sounds
> reasonable, the advantage is minimal locking scope. I'll see if it's
> possible to access msm from the application (different classloaders
> hierarchy in tomcat, there are some subtle difficulties with this).
I just checked that it's possible to implement such a locking, I have
a simple version of this working locally.
Things I'd also implement would be some kind of version check, so that
the session (which might be already loaded/used) is checked for a
modification in memcached and is refreshed/reloaded if needed. The
lock method could directly return the fresh/locked session.
Cheers,
Martin