twisted.web session balancing

[Jacob Feisley]

Hi everyone,

I have a twisted.web application that I wish to load balance, however,
I am not coming up with many resources on good ways to balance with
sessions. I assume the two approaches are:

1. Keep the session's in a common place or sync'd between the t.web
instances
2. Have the load balancer inspect the session and always pass the
client to the same t.web instance serving said session.

One case to worry about in 2 is if the instance goes down, do you just
lose your session? I guess this is still better than non load
balancing where your whole app just dies.

Thanks for any ideas/insights you all can provide!


Cheers,

Jacob Feisley

[Jehiah Czebotar]

On Wed, Oct 22, 2008 at 12:59 PM, Jacob Feisley <jacob....@gmail.com> wrote:
> ...

> I have a twisted.web application that I wish to load balance, however,
> I am not coming up with many resources on good ways to balance with
> sessions. I assume the two approaches are:
>
> 1. Keep the session's in a common place or sync'd between the t.web
> instances
> 2. Have the load balancer inspect the session and always pass the
> client to the same t.web instance serving said session.

> ...

I think the normal approach is some variation of #1. keep the sessions
in a common place.

I use memcached as my primary session store, with a backup copy stored
on a common drbd replicated filesystem partition that is mounted on
all my app servers. This means that if it ever can't get the session
from memcache (ie the cache was flushed), it will retrieve it from the
filesystem (slightly slower), and if even that fails, i can re-create
a basic session (authentication/user info only) from an encrypted
cookie from the client.

This allows the client to always get a working session regardless of
which app server they get connected to. It also means that i can use
leastconns load balancing which is exactly what i want.

--
Jehiah

[Jacob Feisley]

Very interesting approach. How did you go about replacing the session
store in twisted? Did you simply override it with your own
implementation or is there some clever way to tie in a different
session store that I am missing?

-- Jacob

On Oct 22, 1:07 pm, "Jehiah Czebotar" <jeh...@gmail.com> wrote:

[Jehiah Czebotar]

in my case I load balance lots of things, but none of those are
twisted based web applications (well one is a twisted based xmlrpc
server, but no sessions there).

hopefully my approach to sessions is generic enough that you can write
the code necessary to change how twisted sessions are handled. I can't
give any tips on that though, and it's probably a question better
suited to a twisted specific list more than this load balancing list.

--
Jehiah