Weird session

[Seth Buntin]

I am running Apache 2.0.59 and mod_python on Windows. When I run the
application it looks like the session variables are being set. If I
go straight in to the development server everything is great! What am
I missing?

Thanks.

Seth

[Seth Buntin]

aren't working...sorry

[Malcolm Tredinnick]

Almost impossible to tell from this information, unfortunately. Do you
mean that Django's middleware is not running properly, so your code does
not see the effects of receiving the session? Or do you mean that the
session cookie is not being passed back to the client?

You may need to use something like "curl -I ..." to see what is being
sent back and forth (or the handy Web developer's extension in Firefox)
to determine the latter.

Malcolm

[Seth Buntin]

I have this:

request.session["order_items"] = []

and on certain views I have:

request.session["order_items"].append(order_item.id)

In development mode this works fine. Even when I use Firebug I see
everything being sent to the specific views and other areas of the
site are being updated correctly (they don't rely on the session
variable) and are receiving the variables I send via AJAX. It is
acting like the session variable is just being passed over. Kinda
like the session isn't being saved but I have
SESSION_SAVE_EVERY_REQUEST = True in my settings file.

Seth

On Mar 1, 9:04 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

Hi Seth,

On Fri, 2007-03-02 at 03:35 +0000, Seth Buntin wrote:
> I have this:
>
> request.session["order_items"] = []
>
> and on certain views I have:
>
> request.session["order_items"].append(order_item.id)
>
> In development mode this works fine. Even when I use Firebug I see
> everything being sent to the specific views and other areas of the
> site are being updated correctly (they don't rely on the session
> variable) and are receiving the variables I send via AJAX. It is
> acting like the session variable is just being passed over. Kinda
> like the session isn't being saved but I have

> SESSION_SAVE_EVERY_REQUEST = True in my settings file.'

Thanks for the extra information. Makes things a little clearer.

Slight stab in the dark here, but whilst I think about it some more,
this may provide some clues for you to think about: one difference
between the development environment and mod_python is threadedness. In
the dev environment, everything is being run in a single thread, one
step at a time. In the mod_python setup, different threads could be
handling different portions of your processing each time.

I'm wondering if this may lead to race conditions along the lines of:

thread A creates a session

thread B creates a session

thread A stores "foo = 6" in session

thread B stores "blah = 1" in its session

thread A stores session (which contains foo=6)

thread B stores session (which does NOT have foo=6)

might be happening if you are doing lots of partial updates.

Like I said, this is just a gut feeling of something to look at based on
one known difference between the two environments. Working out whether
I'm on crack or not may require rummaging through to code to work out
exactly who is saving what when. In the meantime, I'll let it bounce
around in my head a bit and see if anything else pops into mind.

Regards,
Malcolm

[Seth Buntin]

That probably isn't too far off. One time it did work and it was
weird so I might have actually got one of the threads to fire
correctly. Is there a way to limit the amount of threads mod_python
uses?

Seth

On Mar 1, 9:48 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Graham Dumpleton]

On Mar 2, 2:57 pm, "Seth Buntin" <sethtr...@gmail.com> wrote:
> That probably isn't too far off. One time it did work and it was
> weird so I might have actually got one of the threads to fire
> correctly. Is there a way to limit the amount of threadsmod_python
> uses?

The threads are not created by mod_python, they are created by Apache.
For each client request Apache will handle it in a separate thread
taken from a pool of threads. If the response for a request needs to
come from an application running under mod_python, then the call
through to mod_python/Python simply uses the existing thread that
Apache created.

The only way to limit the number of threads is by changing the
configuration of Apache, but on Windows this would be a silly thing to
do as there is only one Apache process and if you limit your threads
to 1 to avoid perceived threading issues, all client requests will
then effectively be serialised which would impact performance and
would screw badly with AJAX applications that may want to have
multiple concurrent requests going at the same time.

> > I'm wondering if this may lead to race conditions along the lines of:
>
> > thread A creates a session
>
> > thread B creates a session
>
> > thread A stores "foo = 6" in session
>
> > thread B stores "blah = 1" in its session
>
> > thread A stores session (which contains foo=6)
>
> > thread B stores session (which does NOT have foo=6)
>
> > might be happening if you are doing lots of partial updates.

This doesn't make sense. Normally the way sessions work, a second
request is usually locked out until the first request has finished
with the session object and released it. Thus there can be no
overlapping changes unless you are doing silly things like unlocking/
locking them yourself during a request. At least this is how
mod_python sessions work. How Django sessions (which don't actually
use mod_python sessions) work I don't know.

If there is some problem with locking and concurrent requests changing
the same actual session object at the same time it should be easy to
pick up by adding into your code suitable debug logging at start/end
points related to manipulation of session object and possibly in
between, which shows identity of request/thread doing the
manipulation.

Graham

[Seth Buntin]

What type of debug code could I use?

Seth

[Malcolm Tredinnick]

On Thu, 2007-03-01 at 20:17 -0800, Graham Dumpleton wrote:
[...]

I wrote:
> > > I'm wondering if this may lead to race conditions along the lines of:
> >
> > > thread A creates a session
> >
> > > thread B creates a session
> >
> > > thread A stores "foo = 6" in session
> >
> > > thread B stores "blah = 1" in its session
> >
> > > thread A stores session (which contains foo=6)
> >
> > > thread B stores session (which does NOT have foo=6)
> >
> > > might be happening if you are doing lots of partial updates.
>
> This doesn't make sense.

Actually it makes a lot of sense now that I look at the code.

> Normally the way sessions work, a second
> request is usually locked out until the first request has finished
> with the session object and released it. Thus there can be no
> overlapping changes unless you are doing silly things like unlocking/
> locking them yourself during a request. At least this is how
> mod_python sessions work. How Django sessions (which don't actually
> use mod_python sessions) work I don't know.

Just to keep things on the right track for debugging Set's problem, I'll
note that this isn't correct for Django sessions. As can be seen from
the code in django/contrib/sessions/middleware.py and models.py, we
don't do any cross-thread or cross-process locking when creating a
session instance. Maybe we well should be doing something like that for
people who want to do simultaneous updates -- as might happen in an AJAX
driven site -- but, right now, we do not.

A lot of Django enhancements like this are historically driven by usage
patterns: the whole framework was written for one -- fairly broad --
purpose and is gradually and carefully extended to handle other
reasonable uses as they come up. This would be a good mini-project for
somebody to look at doing if they felt so inclined. A bit of design work
would be required, not just a one-line patch, I supsect, but it's not
going to be of the "cure cancer" level of difficulty.

Regards,
Malcolm

[Seth Buntin]

I would love to tackle this issue, since it is to my advantage, but
really don't know where to start. Any ideas?

Seth

On Mar 1, 10:59 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Nebojša Đorđević]

On Mar 2, 2007, at 04:35 , Seth Buntin wrote:

> I have this:
>
> request.session["order_items"] = []
>
> and on certain views I have:
>
> request.session["order_items"].append(order_item.id)

Maybe is just shoot in the dark, but...

Looking at SessionWrapper.__setitem__ your request.session
["order_items"].append(order_item.id) newer sets session modified
flag because __setitem__ newer gets called. I was bitten many times
by the similar problem.

Try:

request.session["order_items"] = request.session["order_items"] +
[order_item.id]

this way SessionWrapper.__setitem__ will be called and session will
be marked as modified.

Hope that helps.

IIRC Quixote had a set of the special list/tuple/dict/whatever
classes which interact with the "set dirty" machinery and set "dirty"
flag when content is modified (ZODB related).

--
Nebojša Đorđević - nesh, ICQ#43799892
Studio Quattro - Niš - Serbia
http://studioquattro.biz/ | http://code.google.com/p/django-utils/
Registered Linux User 282159 [http://counter.li.org]

[Malcolm Tredinnick]

On Fri, 2007-03-02 at 14:37 +0000, Seth Buntin wrote:
> I would love to tackle this issue, since it is to my advantage, but
> really don't know where to start. Any ideas?

I would start with the Session object itself -- have a look in
django/contrib/sessions/ -- since any general solution will need to work
whether sessions are created by middleware or directly by users.

Regards,
Malcolm

[Seth Buntin]

Well this didn't work:

request.session["order_items"] = request.session["order_items"] +
[order_item.id]

Any other ideas? Is it how Django is implementing sessions or how
mod_python and Apache have their child processes?

Thanks.

Seth