django.channels: "async" for Django

[Andrew Godwin]

Hello everyone,

I've been formulating a plan to decouple Django from the request-response cycle for a couple of years now and after discussing it with some people at DjangoCon Europe I thought it was high time that I actually wrote it up, wrote some code, and solicited feedback.

So here it is. The proposal is very long, and is in a Gist here: https://gist.github.com/andrewgodwin/b3f826a879eb84a70625

As a top-level summary:

 - Fully backwards-compatible
 - No actual "in-process" async like asyncio/greenlets (apart from a couple of optional "interface server" modules where you can pick your async implementation)
 - New base unit of Django work is no longer the view, it is the channel consumer
 - Supports WebSockets, long-polling, offsetting complex tasks outside views and more
 - Code looks and works the same - since it'll be a worker model, there's no need to retrofit any core Django APIs for async stuff.

To help write the proposal and also to help you understand it, I've semi-implemented the proposal over here as a third-party app: https://github.com/andrewgodwin/django-channels

While the goal is to merge this into Django proper, prototyping and iterating outside is much faster and more accessible for everyone involved, and the goal is to get it working on 1.7 and 1.8 (currently only 1.8).

The basic chat example in the README of that project is up and running on this code (and the Redis channel backend) here: http://www.aeracode.org/chat/

Any and all feedback is welcomed, in this thread is probably best. This is by no means a final plan, but I hope it will lead us towards something that will make Django much more suitable for a modern Web.

(I'm about to get on a long flight to Tokyo, so I may not be immediately responsive! And my example might crash, it's only running the websocket server in screen!)

Andrew

[BearXu]

Personally I don't agree with implementing the Channel using in-memory mode first. Writing code to communicate among  sockets will become very difficult for me(ordinary developer). If we see async in Django as a tool instead of a coding style, I think Redis/NoSql/Database backend/ is good enough as a first step.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1upKYO7MxyuRigYphJfCqqj%2BKPp_uhBgymEK2oMtP%2BJMPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Godwin]

The in-memory backend is not meant to enable any new functionality, just allow things to run in-place as it currently works - any cross-socket communication would be disabled in that mode. You can already see this in my beta code - trying to run the WebSocket or Worker processes in memory mode will just fail.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAB%2B2MZygJ10J8_bPFkE8AR%3D1mQ%3D0KKWWvXvoHhnmKaVJzZFBGQ%40mail.gmail.com.

[BearXu]

Sorry, I haven't read the source code. But I basically read the doc. Thus in a multithreaded env, we will use Redis or something to implement the one to one chat function, am I right?

Thus people might be confused with the different channels' definitions.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1urENRO2bFVVncpbF_Dymv0Z1PAVUJxHW7kOScubCg4sJg%40mail.gmail.com.

[BearXu]

It might be a better idea to set the task-pool and intersocket--async into two namespaces. It would become confused if we put them together under the "Channel". Like in Java, Channel is like a wrapped-socket, but here in django-channel, it also means a task queue.

[Andrew Godwin]

The only threading is when you're using runserver - normally I'd suggest this is a multi-process model instead.

In my view, they already are namespaced, as well; the communication/socket channels live under django.wsgi and django.websocket prefixes, whereas the custom ones don't (perhaps we could suggest that people prefix them with their app name). I think enforcing separation beyond this isn't sensible if the behaviour is going to be exactly the same anyway.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAB%2B2MZy5je0f9-UpgaRduC0_hYHrW5w3egGMTbCBF9RUzT%2BVgA%40mail.gmail.com.

[Federico Capoano]

Hi Andrew,

i have been taking a look at your work because I find it interesting and I also think that django should have some core mechanism for integrating websockets.

I was wondering, have you also tried one of the other popular django websocket apps? Have you found any interesting one that inspired your work?

Federico

[Emil Stenström]

I finally found your proposal yesterday and I REALLY think this is the right step forward for Django. Implementing this mindset for one of my current projects I can already see how I could more easily add realtime stuff, and rewrite much of the signaling to make the code easier to reason about.

Great job, I will be following it closely.

[Tino de Bruijn]

Hi Andrew,

From reading your proposal, I really like this idea. I did not got around to trying django-channels yet, but it will in August when I start a new project (it's a PoC, so I can try out new stuff :). I hope to be able to give you more feedback then!

Thanks so far,

Tino

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ebff02a5-4890-4931-a28f-8e54820f140d%40googlegroups.com.