SessionExtension.after_attach hook
Laurence Rowe
transaction integration story has become quite simple:
Session = scoped_session(sessionmaker(bind=engine,
extension=ZopeTransactionExtension()))
Here ZopeTransactionExtension ties the session into zope's transaction
manager on first database access. This works well as the integration
is only invoked for those transactions where the database is actually
used.
Now that other zope modules have started to depend on zope.sqlalchemy,
it's become apparent that I've missed a corner case: when you create a
new object you must make sure that the database is contacted or the
transactions will not be tied together. This extract from the
z3c.dobbin doctest demonstrates it:
{{{
>>> session.save(album)
>>> session.save(vinyl)
>>> session.save(cd)
We must actually query the database once before proceeding; this seems
to be a bug in ``zope.sqlalchemy``.
>>> results = session.query(album.__class__).all()
>>> import transaction
>>> transaction.commit()
}}}
Without that query (or an explicit flush or whatever),
transaction.commit() does nothing, as the SessionExtension.after_begin
hook is never invoked.
To fix zope.sqlalchemy I need some way of being notified when a
session is first used. I _think_ the only cases where after_begin is
not called are during an add, delete or merge. I'd like to have
another hook that gets called before or after these operations. Would
it be possible to add an after_attach method to SessionExtension? Or
can anyone think of another way to achieve this?
Laurence
Index: test/orm/session.py
===================================================================
--- test/orm/session.py (revision 4906)
+++ test/orm/session.py (working copy)
@@ -889,18 +889,20 @@
log.append('after_flush_postexec')
def after_begin(self, session, transaction, connection):
log.append('after_begin')
+ def after_attach(self, session, state):
+ log.append('after_attach')
sess = create_session(extension = MyExt())
u = User(name='u1')
sess.add(u)
sess.flush()
- assert log == ['before_flush', 'after_begin', 'after_flush',
'before_commit', 'after_commit', 'after_flush_postexec']
+ assert log == ['after_attach', 'before_flush', 'after_begin',
'after_flush', 'before_commit', 'after_commit',
'after_flush_postexec']
log = []
sess = create_session(autocommit=False, extension=MyExt())
u = User(name='u1')
sess.add(u)
sess.flush()
- assert log == ['before_flush', 'after_begin', 'after_flush',
'after_flush_postexec']
+ assert log == ['after_attach', 'before_flush', 'after_begin',
'after_flush', 'after_flush_postexec']
log = []
u.name = 'ed'
Index: lib/sqlalchemy/orm/session.py
===================================================================
--- lib/sqlalchemy/orm/session.py (revision 4906)
+++ lib/sqlalchemy/orm/session.py (working copy)
@@ -1283,6 +1283,8 @@
state.session_id, self.hash_key))
if state.session_id != self.hash_key:
state.session_id = self.hash_key
+ if self.extension is not None:
+ self.extension.after_attach(self, state)
def __contains__(self, instance):
"""Return True if the instance is associated with this
session.
Index: lib/sqlalchemy/orm/interfaces.py
===================================================================
--- lib/sqlalchemy/orm/interfaces.py (revision 4906)
+++ lib/sqlalchemy/orm/interfaces.py (working copy)
@@ -300,6 +300,12 @@
`transaction` is the SessionTransaction. This method is
called after an
engine level transaction is begun on a connection.
"""
+
+ def after_attach(self, session, state):
+ """Execute after an instance is attached to a session.
+
+ This is called after an add, delete or merge.
+ """
class MapperProperty(object):
Michael Bayer
this depends on what your definition of "used" means. A save() is a
passive operation that does not interact with the database at all.
You'd have to explain to me what it is Zope needs to do with objects
at the moment of save() (and what about update? in 0.5 they are merged
into just add()), that cannot be handled at flush or query execution
time. Hooking into individual save() operations can be expensive
since a save() operation can cascade to any number of child items.
OTOH, if you need zope to wake up whenever anything on the Session at
all is called, I'd suggest a wrapper or subclass of Session. The
"sessionmaker()" concept makes this very easy since you get to plug in
whatever session factory you want using the "class_" parameter.
Laurence Rowe
> this depends on what your definition of "used" means. A save() is a
> passive operation that does not interact with the database at all.
> You'd have to explain to me what it is Zope needs to do with objects
> at the moment of save() (and what about update? in 0.5 they are merged
> into just add()), that cannot be handled at flush or query execution
> time. Hooking into individual save() operations can be expensive
> since a save() operation can cascade to any number of child items.
the Zope transaction, so that Zope knows to commit the SQLAlchemy
transaction when it's transaction is committed. At the moment this
join is happening only on first database access, but it's possible to
add an instance without triggering any database access, with the
result that the sqlalchemy session is not committed when zope does
it's transaction.commit().
The same is true of update, merge and delete as far as I can see, but
all these methods end up calling attach, so the patch adds the hook
there.
I would rather only hook in once, but adding the hook into every
attach seems the easiest way to do it. The 'have I hooked in already'
logic moves to the SessionExtension, but I don't think this is a
problem. In zope.sqlalchemy this overhead is only the method call and
one dictionary lookup.
> OTOH, if you need zope to wake up whenever anything on the Session at
> all is called, I'd suggest a wrapper or subclass of Session. The
> "sessionmaker()" concept makes this very easy since you get to plug in
> whatever session factory you want using the "class_" parameter.
SQLAlchemy rather than a magic zope layer in between.
Laurence
Michael Bayer
On Jul 15, 2008, at 7:29 AM, Laurence Rowe wrote:
> On the moment of save, I need to join the SQLAlchemy transaction to
> the Zope transaction, so that Zope knows to commit the SQLAlchemy
> transaction when it's transaction is committed. At the moment this
> join is happening only on first database access, but it's possible to
> add an instance without triggering any database access, with the
> result that the sqlalchemy session is not committed when zope does
> it's transaction.commit().
Oh, OK, the Zope transaction is already in progress. The Session gets
used, possibly just save() and nothing else, then the Zope transaction
is committed.
> I would rather only hook in once, but adding the hook into every
> attach seems the easiest way to do it. The 'have I hooked in already'
> logic moves to the SessionExtension, but I don't think this is a
> problem. In zope.sqlalchemy this overhead is only the method call and
> one dictionary lookup.
OK, the patch is pretty harmless for 0.5, but it does make an
imposition as to how objects are "attached" to the Session now. In
particular, objects which are loaded from a database result set are
not attached through this route, which means we have to always ensure
that _attach() doesn't get called by any other avenue than those
defined (i.e. add(), delete(), merge(), and equivalents). We just
had someone else building an on-save() validator so two use case
requests in a week generally imply we should do something about it so
its in r4932, however the method receives the instance and not the
state, since the state is generally an SA-internal construct.
>
> I'd like to avoid this if I possibly can so that people are just using
> SQLAlchemy rather than a magic zope layer in between.
>
Though I am curious why a Session isn't part of the Zope transaction
from it's inception ? Why not just link the creation of the Session
itself to the transaction association process ? The user needs to
build the sessionmaker() in a Zope-aware way regardless.
>
Laurence Rowe
On Jul 15, 3:56 pm, Michael Bayer <zzz...@gmail.com> wrote:
> On Jul 15, 2008, at 7:29 AM, Laurence Rowe wrote:
>
> > On the moment of save, I need to join the SQLAlchemy transaction to
> > the Zope transaction, so that Zope knows to commit the SQLAlchemy
> > transaction when it's transaction is committed. At the moment this
> > join is happening only on first database access, but it's possible to
> > add an instance without triggering any database access, with the
> > result that the sqlalchemy session is not committed when zope does
> > it's transaction.commit().
>
> Oh, OK, the Zope transaction is already in progress. The Session gets
> used, possibly just save() and nothing else, then the Zope transaction
> is committed.
> > I would rather only hook in once, but adding the hook into every
> > attach seems the easiest way to do it. The 'have I hooked in already'
> > logic moves to the SessionExtension, but I don't think this is a
> > problem. In zope.sqlalchemy this overhead is only the method call and
> > one dictionary lookup.
>
> OK, the patch is pretty harmless for 0.5, but it does make an
> imposition as to how objects are "attached" to the Session now. In
> particular, objects which are loaded from a database result set are
> not attached through this route, which means we have to always ensure
> that _attach() doesn't get called by any other avenue than those
> defined (i.e. add(), delete(), merge(), and equivalents). We just
> had someone else building an on-save() validator so two use case
> requests in a week generally imply we should do something about it so
> its in r4932, however the method receives the instance and not the
> state, since the state is generally an SA-internal construct.
>
>
>
> > I'd like to avoid this if I possibly can so that people are just using
> > SQLAlchemy rather than a magic zope layer in between.
>
> Though I am curious why a Session isn't part of the Zope transaction
> from it's inception ? Why not just link the creation of the Session
> itself to the transaction association process ? The user needs to
> build the sessionmaker() in a Zope-aware way regardless.
session maker is only called once per thread, rather than per
transaction. When the Zope transaction starts it is not associated
with any resources, it is the responsibility of each data manager
(e.g. one for the ZODB, the one in zope.sqlalchemy) to join the
transaction when it is first used. Perhaps it would be better to
create new sessions for each transaction, but the downside is that we
have to start managing engines, rather than just create them each
time. z3c.saconfig is starting to offer infrastructure for this though
so maybe we will explore this route in the future.