Confused about meta.Session

[Josh Kelley]

I finally found the root cause of the "MySQL session has gone away"
errors that I posted about in
http://groups.google.com/group/pylons-discuss/browse_thread/thread/c0f1fe15c841303.
I'm using repoze.what.plugins.quickstart, following the example at
http://wiki.pylonshq.com/display/pylonscookbook/Authorization+with+repoze.what:

from admin.model.meta import Session
from admin.model.backoffice import BackOfficeUser

def add_auth(app):
return setup_sql_auth(app, BackOfficeUser, None, None,
Session.
post_login_url='/logged_in', post_logout_url='/
logged_out')

Apparently meta.Session is imported here before it's assigned its
final value by admin.model.init_model, so even though add_auth is
executed after init_model, it gets the pre-init_model version of
Session, so it ends up using its own session, separate from the rest
of the app, that's never removed and never recycled.

I guess I'm confused now as to why meta.Session is set up the way it
is. This approach seems like it can introduce subtle bugs; what's the
best way to avoid bugs like this in the future?

Specific questions:

Should I update the wiki.pylonshq.com article to use meta.Session
instead of importing Session directly?

If using the pre-init_model meta.Session can introduce bugs, then why
not set it to None (so attempts to use it will fail completely)
instead of scoped_session(sessionmaker())?

In general, is "from admin.model import meta" (then write code
referring to "meta.Session") better style than "from admin.model.meta
import Session" (then write code referring to "Session")?

Using "from admin.model.meta import Session" is obviously unsafe in
the repoze.what sample app's auth.py. Are there other places where
"from admin.model.meta import Session" is always safe to use or always
unsafe to use?

Does Python offer a way to have the shorter syntax of "from
admin.model.meta import Session" (so I can write code referring to
"Session") without having to worry about "Session" being reassigned to
something else? (I guess I want a way to import "Session" as a
reference to "meta.Session" instead of importing the value of the
current "meta.Session" reference.)

--
Josh Kelley

[Daniel Holth]

This pitfall can be avoided by using SQLAlchemy properly.

# Session is configured, but never reassigned:
Session = orm.sessionmaker()

def initialize(engine):
Session.configure(bind=engine)

[Mike Orr]

This is the way to go, but isn't Josh doing it already?

The Session class takes care of creating session objects as necessary
and setting their bind attribute. There may be a problem of an
existing session object when Session.configure is called, but you
should be calling initialize() at the very beginning of the
application in environment.py, before the middleware stack has been
set up.

The Pylons SQLAlchemy template used to set Session to None initially,
but some users complained that they couldn't import Session directly
in that case, they had to import meta.session. The Session.configure
solution is recommended by SQLAlchemy's author Mike Bayer, so that's
why Pylons is using it now.

--
Mike Orr <slugg...@gmail.com>

[Josh Kelley]

I see what I was doing now; I had copied my initial code from the
sample project in "The Definitive Guide to Pylons", which does
reassign Session, before I fully understood how the Session object
works.

I'll update my app to follow the current Pylons template's approach.
Thank you both for your help.

--
Josh Kelley

[Daniel Holth]

> This is the way to go, but isn't Josh doing it already?

I did not check the Pylons template. The linked wiki page Josh is
following http://wiki.pylonshq.com/display/pylonscookbook/Authorization+with+repoze.what
sets Session = None.

Originally I thought Josh was accidentally using a different, but
valid, Session and transaction for his application code than the one
that fetched the user. SQLAlchemy doesn't like that and it is
inefficient. Instead, the entire request should use the same Session
for example by placing repoze.tm2 in front of repoze.who.plugins.sa in
the WSGI stack. Paraphrased:

try:

app(environ, start_response)
Session.commit()

except:
Session.rollback()
raise

finally:
Session.remove()

[Daniel Holth]

On the other hand, lib/base.py does correctly clean up SQLAlchemy
sessions. Here is what happens:

wsgi pipeline = sa plugin -> application

sa plugin acccesses session
(if there is no session) the real session is lazily created in a
threadlocal thanks to scoped_session()
look up user
sa plugin calls app

app does its thing
app cleans up that threadlocal session

Charmingly, this code creates the database session in one level of the
call stack and cleans it up at another. repoze.who.plugins.sa cannot
touch the session on egress. The programmer must remember to clean up
the session if he used it in application init, otherwise the first
request will get a slightly-used session.

[Marius Gedminas]

On Mon, Nov 22, 2010 at 10:45:34AM -0800, Josh Kelley wrote:
> I finally found the root cause of the "MySQL session has gone away"
> errors that I posted about in
> http://groups.google.com/group/pylons-discuss/browse_thread/thread/c0f1fe15c841303.
> I'm using repoze.what.plugins.quickstart, following the example at
> http://wiki.pylonshq.com/display/pylonscookbook/Authorization+with+repoze.what:
>
> from admin.model.meta import Session

...

>
> Apparently meta.Session is imported here before it's assigned its
> final value by admin.model.init_model, so even though add_auth is
> executed after init_model, it gets the pre-init_model version of
> Session, so it ends up using its own session, separate from the rest
> of the app, that's never removed and never recycled.

Ah, ouch.

Well, the templates that come with Pylons 1.0 avoid reassigning any
global names in model.meta. Instead, model/__init__.py looks like this:

from (project).model.meta import Session, Base

def init_model(engine):
"""Call me before using any of the tables or classes in the model"""
Session.configure(bind=engine)

(Incidentally, it would be useful to have a sample project created using
the latest templates on the web somewhere. Say, github; updated
automatically whenever you change the template, so that, for example,
people could see the differences between a "standard" Pylons 0.97
project and a Pylons 1.0 one.)

> I guess I'm confused now as to why meta.Session is set up the way it
> is. This approach seems like it can introduce subtle bugs; what's the
> best way to avoid bugs like this in the future?

Do the same thing as above: avoid assignments to globals.

> Specific questions:
>
> Should I update the wiki.pylonshq.com article to use meta.Session
> instead of importing Session directly?

That might be safer, for people who have older projects with the unsafe
init_model. However the new convention seems to be from foo.model.meta
import Session.

Marius Gedminas
--
Cheap, Fast, Good -- pick two.

[Mike Orr]

On Mon, Nov 22, 2010 at 1:02 PM, Marius Gedminas <mar...@gedmin.as> wrote:
>> Should I update the wiki.pylonshq.com article to use meta.Session
>> instead of importing Session directly?

If the article is not using Session.configure, it should be changed.
But it doesn't matter if it imports meta or Session. You can add a
note saying it's valid both ways.

--
Mike Orr <slugg...@gmail.com>