Motor and get_current_user(), async for prepare()

[Meir Kriheli]

Hi,

I'm trying to implement get_current_user using Motor, but can't seem to wrap my head around it.

The app uses multi tenant db's, so each tenant (by Host header) is queried against a central db, get's the connection params, as the db's may be distributed and hosted on different machines, and connects to it using Motor's async connection (once connected it'll be stored and reused).

Since get_current_user() and prepare() are not async, this poses a problem: How can one use async operations for those methods ?

Thanks

--

Meir Kriheli
http://meirkriheli.com

[Meir Kriheli]

Here's a gist with some code to match:

https://gist.github.com/MeirKriheli/5562785

Cheers

[Florian Ludwig]

Hi,

maybe https://groups.google.com/forum/?fromgroups=#!topic/python-tornado/ZftJUVBBd-w this helps?

Best regards,

[Meir Kriheli]

Thanks Florian,

Seen this already, it's seems more of a hack than a solution, and specific to get_current_user.

In the gist above, I'd like also to set the db as an instance attribute, once async connected. prepare() seems to be a natural fit. But needs to be done async.

So the question is, is there a way to turn the methods of the RequestHandler life cycle (like prepare) to async ones ?

Cheers

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[A. Jesse Jiryu Davis]

There's no way to turn prepare() or get_current_user() async. (I hear rumblings from Ben that Tornado will soon allow them to be treated as async if they return a Future.) The get_current_user() solution Florian linked to is a fine one, not a hack IMO—specifically, I believe it's reasonably maintainable so I wouldn't call it a hack.

In your gist, you pass an open MotorClient as 'db' into the application as a keyword arg, so it'll be available to all RequestHandlers as self.settings['db']. Though since what you're passing is a client, not a database, I'd call it 'client', not 'db', and access it as self.settings['client'].

Your gist ignores the global MotorClient you've opened and connects a new MotorClient on every GET request. This is tremendously expensive, it disables connection pooling, etc. You should get a database from the global client, self.settings['client'], instead of creating a new client for every request.

[Meir Kriheli]

Like I've mentioned, the global one is used for the central property.

Each tenant (based on Host header) queries it's settings from "central" and connects to the db defined in there (could be another host, replicaset etc),

those connections are reused (no news one is opened, as it's stored in the tenants attribute, so it should open just once).

[A. Jesse Jiryu Davis]

That's not how it appears in your gist:

https://gist.github.com/MeirKriheli/5562785#file-tornado_motor-L53

get() calls coll() calls db(), which instantiates a MotorClient called cx and yields cx.open().

[Meir Kriheli]

Line 37 should handle it:

https://gist.github.com/MeirKriheli/5562785#file-tornado_motor-L37

If the tenant is already found, it should gen.Return it's db.

But I have a problem the Motor segfaults for me right now, even with a simple test case like:

In [1]: import motor

In [2]: from tornado import gen

In [3]: @gen.coroutine
   ...: def get_db():
   ...:     cx = motor.MotorClient()
   ...:     conn = yield cx.open()
   ...:     raise gen.Re
   ...:     raise gen.Return(conn)
   ...:

In [4]: from tornado import ioloop

In [5]: x = ioloop.IOLoop.current().run_sync(get_db)
Segmentation fault (core dumped)

Any ideas ? This motor updated from git, PyMongo 2.5 and Tornado 3.0.1 (happens with Motor 0.1 as well)

[A. Jesse Jiryu Davis]

Oh, I see. It looks to me like your code will still create multiple MotorClients, when there should be exactly one MotorClient for your entire program. One.

As for the segfault, that can be a symptom of using C extensions from the bson or pymongo modules that were compiled for a different Python version; try uninstalling and reinstalling pymongo.

[Ben Darnell]

On Sun, May 12, 2013 at 10:53 AM, A. Jesse Jiryu Davis <je...@emptysquare.net> wrote:

There's no way to turn prepare() or get_current_user() async. (I hear rumblings from Ben that Tornado will soon allow them to be treated as async if they return a Future.)

Just prepare(), actually.  get_current_user cannot be made asynchronous because its ultimate purpose is to lazily populate self.current_user inside getattr, so it has to be synchronous.  However, with the (also brand new) ability to assign to self.current_user you can simply set the current user in an asynchronous prepare method:

  @gen.coroutine

  def prepare(self):

    self.current_user = yield db.fetch(...)

I've just checked in support for asynchronous prepare() (and as an added bonus, the @asynchronous decorator is no longer necessary on get/post/etc methods if those methods are using @gen.coroutine or @return_future).  Are there other methods besides prepare that should get the same treatment?

-Ben

[A. Jesse Jiryu Davis]

I'd love if RequestHandler.make_static_url checked whether static_handler_class.make_static_url had returned a Future, and if so yielded it.

Background: Motor has a GridFSHandler that's like Tornado's StaticFileHandler but serves files from MongoDB instead of the file system. MongoDB actually tracks the file's MD5 within the database server, so Motor can (asynchronously) retrieve the MD5 from MongoDB instead of computing it from Python.

But I believe this would require some major surgery on tornado.template, because the ultimate goal is for template expressions like:

{{ static_url('foo') }}

...to be optionally async, so motor.GridFSHandler can asynchronously retrieve the MD5 from MongoDB.

[Ben Darnell]

On Sun, May 12, 2013 at 8:32 PM, A. Jesse Jiryu Davis <je...@emptysquare.net> wrote:

I'd love if RequestHandler.make_static_url checked whether static_handler_class.make_static_url had returned a Future, and if so yielded it.

Interesting.  We can't really capture the asynchronousness in RequestHandler.make_static_url, though, so the best we can do is just pass the Future along.  The better candidates for automatic Future detection are methods that are called by the framework rather than the application, ideally either before or after the main body of the request (for example, compute_etag, write_error, log_request).

We could phase in an optionally-asynchronous interface over time, but I don't think it's the right direction for static_url.  I'm actually leaning towards making static_url more heavily cached and precomputed (on the assumption that it's primarily useful for a finite and enumerable set of files).  GridFSHandler could have its own asynchronous make_gridfs_url without the same backwards-compatibility considerations, although the template issues complicate matters.

 

Background: Motor has a GridFSHandler that's like Tornado's StaticFileHandler but serves files from MongoDB instead of the file system. MongoDB actually tracks the file's MD5 within the database server, so Motor can (asynchronously) retrieve the MD5 from MongoDB instead of computing it from Python.

But I believe this would require some major surgery on tornado.template, because the ultimate goal is for template expressions like:

{{ static_url('foo') }}

...to be optionally async, so motor.GridFSHandler can asynchronously retrieve the MD5 from MongoDB.

The template changes aren't actually that bad (https://github.com/facebook/tornado/pull/553/files), but I'm concerned about opening the door to even more complexity and logic inside templates.  

-Ben

[Meir Kriheli]

Hi Jesse,

On Sun, May 12, 2013 at 7:48 PM, A. Jesse Jiryu Davis <je...@emptysquare.net> wrote:

Oh, I see. It looks to me like your code will still create multiple MotorClients, when there should be exactly one MotorClient for your entire program. One.

The code needs to connect to several different mongodb hosts (each hosting different dbs). A single MotorClient can hold connections to multiple different hosts ?

As for the segfault, that can be a symptom of using C extensions from the bson or pymongo modules that were compiled for a different Python version; try uninstalling and reinstalling pymongo.

Duh, forgot pymongo compiled extensions, alas, didn't help. I've removed pymongo and motor, re-installed both, made sure in the output that it recompiles the extensions for pymongo, and motor was pulled from the git, still getting segfaults. While running tests, it sometimes segfaults, and sometimes fails with the following traceback:

    Traceback (most recent call last):
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/web.py", line 1055, in _stack_context_handle_exception
        raise_exc_info((type, value, traceback))
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/stack_context.py", line 263, in _nested
        yield vars
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/stack_context.py", line 236, in wrapped
        callback(*args, **kwargs)
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 551, in inner
        self.set_result(key, result)
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 480, in set_result
        self.run()
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 508, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/meir/devel/venv/akashi2/akashi/frontend/filtering.py", line 61, in get
        db = yield get_async_db()
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 500, in run
        next = self.yield_point.get_result()
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 402, in get_result
        return self.runner.pop_result(self.key).result()
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/concurrent.py", line 129, in result
        raise_exc_info(self.__exc_info)
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 228, in wrapper
        runner.run()
      File "/home/meir/devel/venv/akashi2/lib/python2.7/site-packages/tornado/gen.py", line 510, in run
        yielded = self.gen.send(next)
      File "/home/meir/devel/venv/akashi2/akashi/base/db.py", line 157, in get_async_db
        conn = yield cx.open()
      File "/home/meir/devel/venv/akashi2/src/motor/motor/__init__.py", line 750, in open
        self._open(callback=callback_from_future(future))
      File "/home/meir/devel/venv/akashi2/src/motor/motor/__init__.py", line 768, in _open
        gr.switch()
    SystemError: error return without exception set

Thanks for the help so far.

 

 

[A. Jesse Jiryu Davis]

Oh, I didn't understand you're connecting to several hosts. A MotorClient can only connect to one host.

Try recompiling greenlet?

[Meir Kriheli]

Hi Ben,

On Mon, May 13, 2013 at 3:06 AM, Ben Darnell <b...@bendarnell.com> wrote:

On Sun, May 12, 2013 at 10:53 AM, A. Jesse Jiryu Davis <je...@emptysquare.net> wrote:

There's no way to turn prepare() or get_current_user() async. (I hear rumblings from Ben that Tornado will soon allow them to be treated as async if they return a Future.)

Just prepare(), actually.  get_current_user cannot be made asynchronous because its ultimate purpose is to lazily populate self.current_user inside getattr, so it has to be synchronous.  However, with the (also brand new) ability to assign to self.current_user you can simply set the current user in an asynchronous prepare method:

  @gen.coroutine

  def prepare(self):

    self.current_user = yield db.fetch(...)

Yep, prepare() is the place I've aimed for.

 

I've just checked in support for asynchronous prepare() (and as an added bonus, the @asynchronous decorator is no longer necessary on get/post/etc methods if those methods are using @gen.coroutine or @return_future).  Are there other methods besides prepare that should get the same treatment?

Async prepare() covers the use cases I can think about.

Thanks..

[Meir Kriheli]

Remove and re-installed:

 $ pip uninstall greenlet
Uninstalling greenlet:
  /home/meir/devel/venv/akashi2/include/site/python2.7/greenlet.h
  /home/meir/devel/venv/akashi2/lib/python2.7/site-packages/greenlet-0.4.0-py2.7.egg-info
  /home/meir/devel/venv/akashi2/lib/python2.7/site-packages/greenlet.so
Proceed (y/n)? y
  Successfully uninstalled greenlet
(akashi2)meir@metallix ~/devel/venv/akashi2/akashi $ pip install greenlet
Downloading/unpacking greenlet
  Downloading greenlet-0.4.0.zip (72kB): 72kB downloaded
  Running setup.py egg_info for package greenlet
   
Installing collected packages: greenlet
  Running setup.py install for greenlet
    building 'greenlet' extension
    gcc -pthread -fno-strict-aliasing -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -I/usr/include/python2.7 -c greenlet.c -o build/temp.linux-x86_64-2.7/greenlet.o
    gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro build/temp.linux-x86_64-2.7/greenlet.o -L/usr/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/greenlet.so
    Linking /home/meir/devel/venv/akashi2/build/greenlet/build/lib.linux-x86_64-2.7/greenlet.so to /home/meir/devel/venv/akashi2/build/greenlet/greenlet.so
   
Successfully installed greenlet
Cleaning up...

Still Segfaults, with the same exception in my previous post.

[A. Jesse Jiryu Davis]

Not sure what to tell you about this segfault—Motor itself has no C code, so the bug must be in one of the other packages' extension modules or CPython itself. I've had no other reports of segfaults related to Motor. That "SystemError: error return without exception set" indicates a bug in some extension module—a C function has returned NULL but hasn't called an error method like PyErr_SetString().

If you do "ulimit -c unlimited" before running your program you should get a core dump, which may help (although I personally have no experience debugging core dumps).

[A. Jesse Jiryu Davis]

Meir, someone in the MongoDB-User mailing list is also seeing this "SystemError: error return without exception set" error:

https://groups.google.com/d/msg/mongodb-user/vKkEmBFiznQ/gwkxtRjik3AJ

Did you ever figure out how to fix this?

Hi Jesse,

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Meir Kriheli
http://meirkriheli.com

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Meir Kriheli
http://meirkriheli.com

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Meir Kriheli
http://meirkriheli.com

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Meir Kriheli
http://meirkriheli.com

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Meir Kriheli
http://meirkriheli.com

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

[Meir Kriheli]

Hi Jesse,

Nope, no resolution. I've noticed that greenlet had some updates to their git repo for gcc 4.8 (that's what ArchLinux carries this days), tried that too, didn't help.

I'm suspecting g_handle_exit() at greenlet.c as it deson't check just for NULL without PyExc_GreenletExit (which maybe something abnormal, but looks like it happens), but nothing definitive yet.

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Meir Kriheli]

Looks like it's related to this bug:
https://github.com/python-greenlet/greenlet/issues/28

As mentioned, installed from master, but still (non-consistent) segfaults.

Cheers

[A. Jesse Jiryu Davis]

Thanks, that's very useful.

[Roey Berman]

I had the same issue with ArchLinux.

The solution metioned here https://github.com/python-greenlet/greenlet/issues/28#issuecomment-16496238 worked for me.

CFLAGS="-O2 -fno-tree-dominator-opts" pip install -I greenlet

[A. Jesse Jiryu Davis]

Thanks Roey and Meir, looks like we have a cause and a solution now. =) Meir, can you confirm here that installing greenlet from its latest version on master fixes the issue? (I see you're in the middle of a conversation w/ the greenlet maintainers in the issue 28 comments.)

[Meir Kriheli]

Hi,

As replied to greenlet's #28, the issue seems to be resolved.

Now to wait for Motor 0.2 and Tornado with async prepare() support to test the new stack, any Ideas regarding the schedule of those ?

Thanks.

[A. Jesse Jiryu Davis]

Motor 0.2: in less than a month. It's not far from done, but I have to concentrate on PyMongo work at the moment.

Thanks for the heads-up about Greenlet. Looks like they're going to release 0.4.1 with the fix real soon now:

https://github.com/python-greenlet/greenlet/commit/259ebd7c