WSGI implementation compatible with asyncio?

[Victor Stinner]

Hi,

Yesterday, the keystone project (authentication) of OpenStack
discussed in a meeting replacing eventlet with Trollius. It looks like
keystone server is currently using routes+WebOb for WSGI, with
eventlet. I don't understand how eventlet is used, it doesn't look to
be used directly.

I would like to know which WSGI implementations are compatible with
Tulip/asyncio (and maybe also Trollius!).

If a WSGI implementation is not designed for asyncio, is it still
possible to run asyncio tasks in an handler?

Logs of the meeting, search "#topic Remove Eventlet + new non-httpd wsgi":

http://eavesdrop.openstack.org/meetings/keystone/2014/keystone.2014-03-25-18.02.log.html

Victor

[Aymeric Augustin]

2014-03-26 12:41 GMT+01:00 Victor Stinner <victor....@gmail.com>:

I would like to know which WSGI implementations are compatible with
Tulip/asyncio (and maybe also Trollius!).

As far as I understand, WSGI as defined by PEP 3333 is incompatible

with asyncio. The WSGI callable would have to become a coroutine for

asyncio to do something useful and the specification doesn't allow that.

At this point, from what I've seen, authors of web applications tend to

separate their sync and async services because the performance

characteristics are widely different.

As far as I know and judging from past attempts, a PEP adding async

support to WSGI would be met with fierce resistance. (I had started

writing one but I gave up after reading discussions on the topic.)

If a WSGI implementation is not designed for asyncio, is it still
possible to run asyncio tasks in an handler?

Most likely yes, by running the main server loop and the asyncio

event loop in different threads.

--
Aymeric.

[Tobias Oberstein]

Am 26.03.2014 14:51, schrieb Aymeric Augustin:
> 2014-03-26 12:41 GMT+01:00 Victor Stinner <victor....@gmail.com

> <mailto:victor....@gmail.com>>:

>
> I would like to know which WSGI implementations are compatible with
> Tulip/asyncio (and maybe also Trollius!).

I've been asked by users for that also already .. like in "can I run
Autobahn/asyncio and WSGI together"? A typical use case would be:

A user posts a form via plain old HTTP to WSGI app. The app does it's
usual stuff, but in addition sends a WebSocket message to notify all
connected clients of some bit of info in the form data posted in real-time.

> As far as I understand, WSGI as defined by PEP 3333 is incompatible
> with asyncio. The WSGI callable would have to become a coroutine for
> asyncio to do something useful and the specification doesn't allow that.

FWIW, Twisted includes a WSGI container that runs the WSGI app bits on a
background thread pool

http://twistedmatrix.com/documents/current/web/howto/web-in-60/wsgi.html

As far as I know, this does not allow to run async. bits in the WSGI app
code. However, what you can do is call the main thread from one of the
background worker threads, and do your async stuff from the main thread
then:

https://github.com/oberstet/scratchbox/blob/master/python/twisted/wsgi/test.py

I am unsure how the other direction would work ..

Then, also FWIW, there is

https://github.com/twisted/klein

which models Flask/Bottle, but directly on top of Twisted Web, which
_does_ allow to do async. stuff

>
> At this point, from what I've seen, authors of web applications tend to
> separate their sync and async services because the performance
> characteristics are widely different.
>
> As far as I know and judging from past attempts, a PEP adding async
> support to WSGI would be met with fierce resistance. (I had started
> writing one but I gave up after reading discussions on the topic.)
>
> If a WSGI implementation is not designed for asyncio, is it still
> possible to run asyncio tasks in an handler?
>
>
> Most likely yes, by running the main server loop and the asyncio
> event loop in different threads.

Ah, yeah. That's what the Twisted WSGI container does ..

Cheers,
/Tobias

>
> --
> Aymeric.

[Luca Sbardella]

On 26 March 2014 11:41, Victor Stinner <victor....@gmail.com> wrote:

I would like to know which WSGI implementations are compatible with
Tulip/asyncio (and maybe also Trollius!).

pulsar has a wsgi server implementation compatible with WSGI 1.0.1 if the wsgi application handler is synchronous. In this respect, it can serve any standard web framework.

In addition, it is able to serve asynchronous wsgi application handlers which conforms to pulsar wsgi specification described here:

http://quantmind.github.io/pulsar/apps/wsgi/async.html#wsgi-async

Both trollius and asyncio are supported thanks to a specialised Task class.

The distribution has an example which shows how to run django and a websocket handler on the same server.

[Gustavo Carneiro]

On 26 March 2014 14:42, Luca Sbardella <luca.sb...@gmail.com> wrote:

On 26 March 2014 11:41, Victor Stinner <victor....@gmail.com> wrote:

I would like to know which WSGI implementations are compatible with
Tulip/asyncio (and maybe also Trollius!).

pulsar has a wsgi server implementation compatible with WSGI 1.0.1 if the wsgi application handler is synchronous. In this respect, it can serve any standard web framework.

In addition, it is able to serve asynchronous wsgi application handlers which conforms to pulsar wsgi specification described here:

http://quantmind.github.io/pulsar/apps/wsgi/async.html#wsgi-async

This looks great.

If the WSGI guys don't like asynchronous frameworks, then we need our own standard along these lines, IMHO.  Nice job! :-)

--

Gustavo J. A. M. Carneiro

Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert