SESSION_SAVE_EVERY_REQUEST creates new session?

797 views
Skip to first unread message

Peter Rowell

unread,
Jan 4, 2008, 5:48:34 PM1/4/08
to Django users
It appears that setting SESSION_SAVE_EVERY_REQUEST = True is causing a
new session to be created on each page access.

Background:

I was doing some fiddling with a 'session browser' so that we can get
an idea of how people are dealing with our new site. I set this
config value because I wanted the timestamp to represent the most
recent page fetch. (In another thread someone had a method for caching
a dict of user/timestamp, but it would seem like this method should be
quite usable.)

After taking a week off from working on this, I couldn't figure out
why formatting the output of Session.objects.all() was taking so long.
I popped into phpMyAdmin and was amazed to find that we have over
290,000 sessions.

This site normally gets about 4000 unique visitors a day, so this
number was over the Moon. I limited my view to the most recent 200
sessions (so it would not crash) and was surprised to see that I had
many sessions for a single user when I would have expected only one.
For example, in less than a minute this user had at least 8 sessions
created for her.

2008-01-04 13:44:59 kirsten d01ac014bdd9f4ec8eb9cd8e2186f6d8
2008-01-04 13:44:58 kirsten 0c06dc7c91e07b114ef1c4e7094f5de0
2008-01-04 13:44:48 kirsten db1997ac362a61355f6112d3ca145901
2008-01-04 13:44:44 kirsten be54f80dfe008b390c2afeb3eb9bd8c7
2008-01-04 13:44:36 kirsten a21147e03b2b73fc9112c4ba3fc4b1f5
2008-01-04 13:44:27 kirsten e50affc7642530c038b6331855fea006
2008-01-04 13:44:18 kirsten c95ba03513b916c47e516cb8ad483236

In phpMyAdmin I kept refreshing the Structure window on django_session
and saw the number of rows going up by dozens at a time. Setting it to
False + restart caused everything to calm down. I set it to True and
sessions were again being created on each page. Set to False, and
things were normal.

I looked at contrib/sessions/*.py and the only place it referenced
this value (middleware.py:process_response) looked completely
reasonable.

Does this ring a bell for anyone?

I am running trunk/6688 on an RHEL 4 box, Python 2.5.1, Apache 2.0.52,
mod_python 3.3.1.

TIA,
Peter

SmileyChris

unread,
Jan 4, 2008, 8:22:59 PM1/4/08
to Django users
Hi Peter,

I've been delving around in that code recently. The only thing I can
think of is that your cookies aren't getting received (perhaps a
misconfigured COOKIE_PATH or one of the other related settings?) so a
new session is getting created and sent each request.

As a side note, I've been thinking that the SAVE_EVERY_REQUEST method
could be optimized to only save sessions which have been modified or
are not empty. Not that it will fix your problem, just wanted to
mention it.

Peter Rowell

unread,
Jan 5, 2008, 12:33:10 AM1/5/08
to Django users
> only save sessions which have been modified
Actually, the code in the sessions/middleware is:
if modified or settings.SESSION_SAVE_EVERY_REQUEST:
so the whole point is to save it whether it's modified or not.

But then you mentioned:
> perhaps a misconfigured COOKIE_PATH

Does "not configured" count as misconfigured? :-)
Sheesh, do I ever feel stoopid.

Although I have settings up the wazoo, I find I have zero, zip, and
nada regarding SESSION_COOKIE_anything.

Looking at conf/global_settings.py, I see these defaults:

SESSION_COOKIE_AGE is 2 weeks, fine.
SESSION_COOKIE_SECURE is False, that's OK.
SESSION_COOKIE_PATH = '/' seems perfectly reasonable.

SESSION_COOKIE_DOMAIN is .... None? Gaack!
How does it even work with None?!

I just checked the site cookies with WebDeveloper Toolbar and the
sessionid Host looks perfectly valid:

Name sessionid
Value 12345ae9eaa7111fddaf0fbfe1f9310e
Host www.example.com
Path /
Secure No
Expires Fri, 18 Jan 2008 21:47:22 GMT

So, that doesn't seem to be a problem ... is it?

So, I'm back to ... what's with all of the rapid session creation?

SmileyChris

unread,
Jan 5, 2008, 2:51:19 AM1/5/08
to Django users


On Jan 5, 6:33 pm, Peter Rowell <hedron...@gmail.com> wrote:
> > only save sessions which have been modified
>
> Actually, the code in the sessions/middleware is:
> if modified or settings.SESSION_SAVE_EVERY_REQUEST:
> so the whole point is to save it whether it's modified or not.

This wasn't related to your issue, just wanted to put out something
I'd been chewing on.
Specifically, there's no point in saving a session if it's empty and
hasn't been modified.

>
> But then you mentioned:
>
> > perhaps a misconfigured COOKIE_PATH
>
> Does "not configured" count as misconfigured? :-)
> Sheesh, do I ever feel stoopid.

No, I was just wondering whether you had set some weird setting -- the
defaults work fine for most.

> I just checked the site cookies with WebDeveloper Toolbar and the
> sessionid Host looks perfectly valid:
>
> So, that doesn't seem to be a problem ... is it?
>
> So, I'm back to ... what's with all of the rapid session creation?

Well if it's creating the cookie fine, then I'm not really sure!

Andrew

unread,
Jan 6, 2008, 3:12:38 AM1/6/08
to Django users
We're looking into this as well! 10K session rows in one day. Have you
noticed the records being in multiples of seven?

Also, in our slow query log we've noticed django performing a
selection on the entire session table with no WHERE clause.

Clues in the puzzle...

Malcolm Tredinnick

unread,
Jan 6, 2008, 7:58:33 AM1/6/08
to django...@googlegroups.com

On Fri, 2008-01-04 at 14:48 -0800, Peter Rowell wrote:
> It appears that setting SESSION_SAVE_EVERY_REQUEST = True is causing a
> new session to be created on each page access.

I was intending to have a quiet evening at home, but James Bennett
convinced me to look at this instead. It should be fixed in [7001]. At
least, one occurrence is fixed in that instance (I managed to get
ever-increasing database entries and now they've stopped).

The situation is a little subtle: by default, we use the same session
cookie name for every Django site. You can change that. In fact, when I
first saw this message, I couldn't repeat the problem because the case I
tested it on had the session cookie being called something unique.

Anyway, with the default name, you will sometimes be submitting a
session cookie that comes from another Django site. The session id won't
match anything in your database, so the subversion version of Django
(with the new pluggable backends ... this shouldn't be a problem in
0.96) will save a new session to the database. However, it wasn't
marking this new session as "modified", so the HTTP response wasn't
sending back a set-cookie header. Thus, on the next request, you'd get
the same bad session id, a *new* session entry would be created...
rinse, wash, repeat.

This would happen even if you weren't specifically using the session on
a particular request (in fact *only* in that case -- because if you
wrote to the session, it was being sent to the user each time). So
[7001] makes sure the new session cookie is sent back to the user as
well.

Please test and see if it fixes your particular case. If not, start
putting debugging prints in django/contrib/sessions/backends/db.py,
particularly in the __init__ and save() methods .. and possibly in
load() as well. This might give you some clues as to where the new
entries are coming from.

Regards,
Malcolm

--
Two wrongs are only the beginning.
http://www.pointy-stick.com/blog/

Malcolm Tredinnick

unread,
Jan 6, 2008, 8:07:56 AM1/6/08
to django...@googlegroups.com

On Sun, 2008-01-06 at 23:58 +1100, Malcolm Tredinnick wrote:
>
> On Fri, 2008-01-04 at 14:48 -0800, Peter Rowell wrote:
> > It appears that setting SESSION_SAVE_EVERY_REQUEST = True is causing a
> > new session to be created on each page access.
>
> I was intending to have a quiet evening at home, but James Bennett
> convinced me to look at this instead. It should be fixed in [7001]. At
> least, one occurrence is fixed in that instance (I managed to get
> ever-increasing database entries and now they've stopped).

I should point out that I couldn't repeat the problem with
SESSION_SAVE_EVERY_REQUEST=True, so what I fixed may be an auxilliary
issue. That's why it needs testing at your end and possibly further
diagnostics there.

Peter Rowell

unread,
Jan 6, 2008, 6:02:19 PM1/6/08
to Django users
Malcolm, please accept my apologies for 1/2 of the problem described
in the original article. I, of all people (I wrote debuggers back in
the 80's), should know that you have to vet your diagnostic tools
before you depend on them.

Short version: The high cookie creation frequency was real (I'm going
to test 7001 tomorrow morning), but the multiple-session-cookies-for-
one-person was not.

On reflection, the multiple-sessions didn't make any sense because it
would have required the user (kristin in this case) to login 8 times
in one minute. That's not going to happen. In the view there was a
variable that was only set when the session variable _auth_user_id was
set. So all sessions from anonymous users looked like they belonged to
the most recent non-anonymous session. Doh.

So I conflated a real problem with a bug of my own.

Thanks for you quick response on the part of this that was real.

Eoin

unread,
Jan 10, 2008, 7:09:51 PM1/10/08
to Django users
I just started to deploy my app to the webserver, grabbed the latest
svn and am having exactly the same issue, its creating a session entry
per page viewed.
But with a difference the users aren't logged in, they are still
viewing anonymously despite the cookie and a session corresponding.

So 7001 didn't seem to do anything. It it helps following this I tried
to change the backend to 'django.contrib.sessions.backends.file' and
its doing exactly the same thing (not logged in / hundreds of files).

Peter Rowell

unread,
Jan 10, 2008, 9:00:43 PM1/10/08
to Django users
Eoin:

Do you have SESSION_SAVE_EVERY_REQUEST = True in your settings.py
file?

I found that with that set to False, the problem went away. From
something Malcolm said, it may also go away if you explicitly set
SESSION_COOKIE_DOMAIN.

Eoin

unread,
Jan 10, 2008, 9:40:06 PM1/10/08
to Django users
I actually found (few hours later) it was a rehash of :
http://groups.google.com/group/django-users/browse_thread/thread/7f4becc6b0a482d0/669b3e3e68f80f84

the md5 of php conflicting with the md5 python ... i disabled php and
all has been great since

Ramiro Morales

unread,
Jan 27, 2008, 1:38:31 PM1/27/08
to django...@googlegroups.com
On Jan 6, 2008 10:58 AM, Malcolm Tredinnick <mal...@pointy-stick.com> wrote:
> [...]

> The situation is a little subtle: by default, we use the same session
> cookie name for every Django site. You can change that. In fact, when I
> first saw this message, I couldn't repeat the problem because the case I
> tested it on had the session cookie being called something unique.
>
> Anyway, with the default name, you will sometimes be submitting a
> session cookie that comes from another Django site. The session id won't
> match anything in your database, so the subversion version of Django
> (with the new pluggable backends ... this shouldn't be a problem in
> 0.96) will save a new session to the database. However, it wasn't
> marking this new session as "modified", so the HTTP response wasn't
> sending back a set-cookie header. Thus, on the next request, you'd get
> the same bad session id, a *new* session entry would be created...
> rinse, wash, repeat.
>
> This would happen even if you weren't specifically using the session on
> a particular request (in fact *only* in that case -- because if you
> wrote to the session, it was being sent to the user each time). So
> [7001] makes sure the new session cookie is sent back to the user as
> well.
>

Just managed to put on Trac some mental notes I had made about topics
being discussed on the mailing lists and tickets I had seen being opened
and found this isue had already been reported as bug #5593. I'm closing it
and leaving a link to this thread there.

Regards,

--
Ramiro Morales

Reply all
Reply to author
Forward
0 new messages