Session gotcha

[Iwan]

Hi there,

We ran into a silly problem which took quite some digging to discover
- I thought I'd share it here and ask for better solutions too.

We have a complicated app which uses a custom session. We set the
custom session by assigning to elixir.session, eg:

elixir.session = scoped_session(sessionmaker(autoflush=True,
transactional=True), scopefunc=custom_scope_func)

However, our elixir Entity-derived classes just happen to be imported
BEFORE that statement executes.

Of course, the result is that they are all using the original
elixir.session (ScopedSession), and not the one we set. And nothing
really breaks to tell you that, so it is hard to find in a complex
app.

Now, we would not want to litter our modules with __session__ or have
to remember to add options to each class. Neither do we want to be
dependent on when what gets imported...

One partial solution to the problem is to NOT assign to
elixir.session, but just to call configure() on it. However, it is
crucial to us to actually set a custom scopefunc and you cannot do
that with configure (unless I am mistaken).

Is it possible to change the scopefunc on a ScopedSession after
creation? Or, alternatively, is it possible to move all classes from
the old elixir.session to the new one we assign?

Regards
-i

[Gaetan de Menten]

On Fri, Oct 9, 2009 at 13:45, Iwan <iw...@reahl.org> wrote:

> We ran into a silly problem which took quite some digging to discover
> - I thought I'd share it here and ask for better solutions too.
>
> We have a complicated app which uses a custom session.  We set the
> custom session by assigning to elixir.session, eg:
>
>  elixir.session = scoped_session(sessionmaker(autoflush=True,
> transactional=True), scopefunc=custom_scope_func)
>
> However, our elixir Entity-derived classes just happen to be imported
> BEFORE that statement executes.
>
> Of course, the result is that they are all using the original
> elixir.session (ScopedSession), and not the one we set.  And nothing
> really breaks to tell you that, so it is hard to find in a complex
> app.

Well that global (ie provided by default) collection, session and
metadata was a poor design decision and this is one of the numerous
problems it brings. At some point, I should really think things
through and see how to get rid of them with minimal upgrade hassle.

> Now, we would not want to litter our modules with __session__ or have
> to remember to add options to each class.

I can understand that...

> Neither do we want to be
> dependent on when what gets imported...

Well, that's the only semi-clean (IMO) option currently available.

> One partial solution to the problem is to NOT assign to
> elixir.session, but just to call configure() on it.  However, it is
> crucial to us to actually set a custom scopefunc and you cannot do
> that with configure (unless I am mistaken).

> Is it possible to change the scopefunc on a ScopedSession after
> creation?

You are correct, as far as I can tell, it's not possible.

> Or, alternatively, is it possible to move all classes from
> the old elixir.session to the new one we assign?

I think you can do it after declaring your entities (but before
setup_all), but that's quite ugly. The following should work (I have
not actually tried):

for entity in elixir.entities:
entity._descriptor.session = XYZ

--
Gaëtan de Menten
http://openhex.org

[Iwan Vosloo]

Thanks Gaetan,

> Well that global (ie provided by default) collection, session and
> metadata was a poor design decision and this is one of the numerous
> problems it brings. At some point, I should really think things
> through and see how to get rid of them with minimal upgrade hassle.

I'd be interested to see what you come up with...

> > Neither do we want to be
> > dependent on when what gets imported...
>
> Well, that's the only semi-clean (IMO) option currently available.

The problem is that we don't always have control over that. And, it is
such an unintuitive place to look for problems. Such a secondary
impact of import order is almost invisible in a way. For me the
logical place to do something about it feels like setup_all (or
something analogous to it)?

We're doing the following now (before setup_all):

elixir.session.configure(autoflush=True, transactional=True)
elixir.session.registry.scopefunc = custom_scope_func

Which seems to be working... although I am uncomfortable fiddling with
the session internals: .registry and its .scopefunc. And, I'm worried
that there are more internal bits I have missed, or that may be
changed in future. Nevertheless, I've decided that I'd rather do this
than try to "move" the Entities from one ScopedSession to another.

Thanks
- i

[Gaetan de Menten]

On Sat, Oct 10, 2009 at 07:47, Iwan Vosloo <iw...@reahl.org> wrote:

>> Well that global (ie provided by default) collection, session and
>> metadata was a poor design decision and this is one of the numerous
>> problems it brings. At some point, I should really think things
>> through and see how to get rid of them with minimal upgrade hassle.
>
> I'd be interested to see what you come up with...

Don't hold your breath though, it might take a while before I fix that
issue, if ever. Of course, a patch in that direction would help ;-).

>> > Neither do we want to be
>> > dependent on when what gets imported...
>>
>> Well, that's the only semi-clean (IMO) option currently available.
>
> The problem is that we don't always have control over that. And, it is
> such an unintuitive place to look for problems. Such a secondary
> impact of import order is almost invisible in a way.  For me the
> logical place to do something about it feels like setup_all (or
> something analogous to it)?
>
> We're doing the following now (before setup_all):
>
> elixir.session.configure(autoflush=True, transactional=True)
> elixir.session.registry.scopefunc = custom_scope_func

Well, I didn't dig that far into SQLAlchemy's code. But I thought this
would work until I saw your post on SQLAlchemy's list... Note that a
session is *not* thread-safe, so whatever you are trying to do needs
to take that into account (and that *might* be the reason your test is
failing).

[Iwan]

Hi Gaetan,

On Oct 13, 2:29 pm, Gaetan de Menten <gdemen...@gmail.com> wrote:
> > We're doing the following now (before setup_all):
>
> > elixir.session.configure(autoflush=True, transactional=True)
> > elixir.session.registry.scopefunc = custom_scope_func
>
> Well, I didn't dig that far into SQLAlchemy's code. But I thought this
> would work until I saw your post on SQLAlchemy's list... Note that a
> session is *not* thread-safe, so whatever you are trying to do needs
> to take that into account (and that *might* be the reason your test is
> failing).

Yes, our different threads are carefully managed not to execute at the
same time. (I know, this sounds silly - but is a side effect of a
special testing environment.)

We have now reverted to importing our own ScopedSession in every
module as __session__, and that does the trick for now. After all,
that is the support elixir now has for this problem, so we ought to
use it instead of trying to push the envelope in some other
direction...

-i