pool and eventlet (and psycopg2)

[Julien Lacroix]

Cheers,

I'm trying to migrate my project, so i can use some of the benefits
Eventlet offers (yum, concurrency).
Right now I've had little success following the (former) approach of the
Nova Project [1].
Is this still the (an) way to handle this? tryed several pool variants
from the docs, but in the end the first querys dies within
a traceback like this:

File
"/usr/local/lib/python2.6/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py",
line 329, in on_connect
extensions.register_type(extensions.UNICODE, conn)
TypeError: argument 2 must be a connection, cursor or None
eventlet db_pool seems to spawn a connection but psycopg2 is able to
handle it. not sure how to debug this.

Somebody using eventlet with sqlalchemy in current versions?

[1]:
http://bazaar.launchpad.net/~rackspace-titan/nova/sqlalchemy-eventlet/view/head:/nova/db/sqlalchemy/session.py
http://docs.sqlalchemy.org/en/latest/core/pooling.html
http://eventlet.net/doc/modules/db_pool.html
http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries

[zz elle]

IMO, Nova/Openstack can not be consider as a reference for using eventlet+sqlalchemy ... indeed according to the way they use SA, they have a really poor understanding of SA ?

But how did you patch the python libraries to support eventlet ?
More precisely how and where in your code do you call eventlet.monkey_patch ?

[Julien Lacroix]

eventlet.patcher.monkey_patch() is called directly after import and shebang, so before all others imports (psycopg2 & sqlalchemy).
i must admit my current approach is just a ripoff of novas (probably poor) implementation, using creator on create_engine doesn't seems to be best practice.

source
https://gist.github.com/2012418  (40 lines)

here is the full traceback btw:
https://gist.github.com/2012316

zz elle

Samstag, 10. März 2012 17:36

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/_-H-IT7FlrIJ.
To post to this group, send email to sqlal...@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.

Julien Lacroix

Samstag, 10. März 2012 13:38

[zz elle]

If i remember well, you need to disable native_unicode with psycopg2 when using a pool of connections with SA.

return sqlalchemy.create_engine(connection, pool_recycle=180,  ...
becomes:
return sqlalchemy.create_engine(connection, use_native_unicode=False, pool_recycle=180,  ...

[Julien Lacroix]

thank you for the advice, it seems to work now :)

zz elle

Samstag, 10. März 2012 20:37

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/xyttqza8fZgJ.

To post to this group, send email to sqlal...@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.

Julien Lacroix

Samstag, 10. März 2012 19:42

eventlet.patcher.monkey_patch() is called directly after import and shebang, so before all others imports (psycopg2 & sqlalchemy).
i must admit my current approach is just a ripoff of novas (probably poor) implementation, using creator on create_engine doesn't seems to be best practice.

source
https://gist.github.com/2012418  (40 lines)

here is the full traceback btw:
https://gist.github.com/2012316

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.