Web framework integration PyMongo

[Chris]

Hi,
I'm trying to integrate with a python wsgi framework (Pylons). What
I've done so far is the following, please let me know if you think of
a better approach. I've run this by the Pylons user group and it
seems to be ok, I just wanted to verify that sharing the connection is
valid.

1. On app launch/load create a single, global pymongo connection
2. On each http request, instantiate a new pymongo database object
(actually a subclass)
3. Then use that database object in my controller actions.

Note: on step 2, I instantiate a new subclass of pymongo's database
per request because I was trying to figure out a way to have the
database object be 'site-aware'. Meaning, db.events.save(doc) will
behave differently depending on the requesting site. (example, if
site is texas.myproject.com, then db.events.save(doc) will save the
doc to the 'texas_events' collection, instead of just 'events'
collection)

::Something like this is run per request::
conn = globals.mongo_conn
# get site info from wsgi environ
site = get_site_context(environ)
controller = Controller()
# pymongo db subclass, site aware
controller.db = SiteDatabase(conn, 'db_name', site)
# call the controller
controller()

# ... Then, in controller action
self.db.events.save(request.POST)
# above line saves the doc to 'subdomain_events' collection

Is it ok to share the global connection object among multiple threads
(requests) ? (seems to be working.)

Other approaches?

[Nicolas Clairon]

Hi !

You might be interested by the MongoKit sources :
http://bitbucket.org/namlook/mongokit/src/
and specialy the pylons_env extension :
http://bitbucket.org/namlook/mongokit/src/tip/mongokit/ext/pylons_env.py.

MongoKit is a ORM wich brings structured schema and validation layer
on top of pymongo.
It has Pylons support and aims to be very simple to use (using KISS
and DRY). You can see
all features on bitbucket. As far as I know, MongoKit is used in
production (you may find those
people in this mailing-list).

[Michael Dirolf]

Chris,
Sharing the Connection instance across threads is fine. You can also
increase the pool_size to take advantage of PyMongo's built in
connection pooling.

- Mike

[Chris]

Mike,
Thanks, I'll look at upping the pool_size.


Nicolas,
Thanks for the suggestion. I did have a look at mongokit and some
of the other ORM layers. It seems that the mongokit documents handle
all of the connection stuff behind the scenes. However, a mongokit
document ties the collection name to the class with the class
attribute ``collection_name``. In my case, I need my collection name
to vary, potentially per request. That is, MongoDocument.save() needs
to behave differently depending on site context. I may have missed
something, but this didn't seem easily doable currently. Plus, I
would like to gain a certain level of familiarity with the lower level
pymongo. I may move to an ORM later, and mongokit's nice feature set
puts it at the top of the list.

I also looked at MongoPylonsEnv and it appears to create a thread-
local Connection instance (if the thread doesn't already have one).
And that made me wonder if I should be doing the same, hence this
post.

Thanks,
Chris