asyncio integration /benefits & performance

[sastry mln]

Hi All,

   

       Recently there was a presentation in our company about node.js and the success stories of using it in walmart and linkedin (it was 2 years old story but news to me). And i started googling about this concept of asychronous I/O and bumped into youtube video about asyncio by Guido

   

   Can and will django use asyncio or any other asyc framework like twisted? will django or django apps benefit by using this new asyncio api?

   Was any performance studies done to compare django with node

    

    Please share your experiences in this area

-regards

[Avraham Serour]

I believe this would be out of scope to django.

nginx+uwsgi to serve your django apps is pretty fast, and don't use much memory or processor, what else do you need?

much of performance problems can be solved by using cache properly

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fd3a0745-3766-4fd3-a761-6176850eff38%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[carlos]

If you need real time application i thing you need read about gevent

http://www.gevent.org/ 

other lecture is socketio and exit third app django

https://github.com/stephenmcd/django-socketio

and here other thing

http://www.pixeldonor.com/2014/jan/10/django-gevent-and-socketio/

Cheers

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFWa6tJHEM7yDoSQy-ZdUgNLsPrWQS6Dh%3DUOdgeihw1RaOP9mw%40mail.gmail.com.

[Russell Keith-Magee]

Hi,

On Wed, Jul 30, 2014 at 1:21 AM, sastry mln <sast...@gmail.com> wrote:

Hi All,

   

       Recently there was a presentation in our company about node.js and the success stories of using it in walmart and linkedin (it was 2 years old story but news to me). And i started googling about this concept of asychronous I/O and bumped into youtube video about asyncio by Guido

   

   Can and will django use asyncio or any other asyc framework like twisted? will django or django apps benefit by using this new asyncio api?

That's a difficult question to answer. It depends entirely on what you mean by "asynchronous".

At the very low level, Async is a way of building *any* system so that there aren't any blocking operations. This potentially means you can handle more throughput with less hardware - but at a price. Building asynchronous systems is much harder than synchronous systems. The simple idea of "do X, then do Y, then do Z" can be quite complex (or at least, convoluted) with an asynchronous system because you have to write it as a series of nested callbacks (which is what Node does), or some other language-level mechanism (which is what Twisted/asyncio does). 

However, if you go asynchronous, *everything* needs to be aynchronous. As soon as *anything* in your stack is synchronous, you effectively lose the benefit of having an async system. This is the biggest holdup in the Django/Python community at the moment - every library that is currently in existence needs to be updated to be "async compatible". This means database drivers, File IO libraries, interfaces with memory stores like Memcache and Redis, and so on. This is, as you might expect, not a small job.

The other way to interpret "asynchronous" is in the context of "web sockets" - that is, building web pages that continuously update based on information from the server. There are many ways to implement this sort of thing, but async frameworks are particularly well suited. 

The two topics are related; in order to build a good web socket system, some asynchronisity is required. However, your entire *stack* doesn't need to be asynchronous in order to add web sockets. 

Aymeric Augustin has dome some exploratory work in this area.

https://github.com/aaugustin/django-c10k-demo

This isn't something that is going to appear in Django 1.8 (or any other soon-to-be-released version), but it is an area that we are aware of and are watching keenly.

   Was any performance studies done to compare django with node 

 

Not that I'd pay any attention to. It's trivial to make a performance study say whatever you want it to by manipulating the benchmark. 

What is indisputable is that Django is able to run some *very* large websites. Mozilla, Disqus, Instagram, Pinterest - these are all Django sites. Yes, they're heavily modified from "stock" Django - but that's because while the performance characteristics of every small website are the same, the performance characteristics of every large website are different. 

Suffice to say - it's entirely possible to make a Django site scale to *very* large traffic levels. If you believe someone who says "Just use Node and it will solve all your performance problems", then I have a bridge to sell you. The reason to use Node, Twisted, Django, or anything else for that matter, shouldn't be made on the basis of a single benchmark. You need to look at the stability of the platform, the community around it, the long term support structures that are in place, and much more. Performance isn't an insignificant concern, but if you know what you're doing, you can make almost *any* web framework perform well if you actually end up with having those sort of problems.

Yours,

Russ Magee %-)