Is Django limited by Python's GIL? If not, are there any benchmarks for multiple connections?

[navinipe]

Hello,

I'm a freelancer just getting introduced to web app programming, and will soon need to create web apps for clients who want a database of info for many of their customers and clients who want an e-commerce website.

I know of Python's GIL, and I assume Django creators would've obviously designed processes to avoid GIL-related issues. This answer speaks a bit about how multiple web requests are handled, and I've come to understand that I have to use external libraries which would use WSGI to perform load balancing. So I'm assuming the load balancer would run multiple instances of my webapp in separate processes to serve client requests, and I'd only have to worry about avoiding race conditions and mutual exclusion issues.

I also searched for benchmarks, and found some Locust tests which seem like Django could handle around 500 requests per second without much sweat. The reason I need to know this, is because I need to know what the threshold is (for number of simultaneous user connections), before I consider installing something as complex as Kubernetes. How many requests can be handled by an ordinary load balancer before resorting to Kubernetes?

I know I'm speaking from inexperience, but these are what anyone would like to know before starting off with building a web-app, and hope the answers that follow this question would be useful for novices.

[Alejandro Garrido Gongora]

https://medium.com/@KonopkaKodes/an-introduction-to-asynchronous-programming-in-python-6809a4385f69

https://docs.djangoproject.com/en/3.0/topics/async/  

https://blog.disqus.com/scaling-django-to-8-billion-page-views 

https://attentionshard.wordpress.com/2011/04/26/scaling-django-to-30000-requests-per-second/

https://www.freecodecamp.org/news/million-requests-per-second-with-python-95c137af319/

There are a few ways to do that, either using third-party modules or driving architectures based on those needs like queuing systems with celery and rabbit :).

Regards.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/13e8f5bb-8f3a-4b82-89d5-f92bfb565f8d%40googlegroups.com.

[navinipe]

This is very helpful. Thanks Alejandro!

[Alejandro Garrido Gongora]

No problem :).

El mar., 26 may. 2020 a las 15:13, navinipe (<navi...@gmail.com>) escribió:

This is very helpful. Thanks Alejandro!

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/40bf9ea7-fa10-4865-af75-c598f7b68fd5%40googlegroups.com.

[graeme]

On Tuesday, May 26, 2020 at 1:33:47 PM UTC+1, navinipe wrote:

Hello,

I'm a freelancer just getting introduced to web app programming, and will soon need to create web apps for clients who want a database of info for many of their customers and clients who want an e-commerce website.

Most clients who hire freelancers do not have hard to solve scaling problems. What do you need to deal with in terms of requests per second and the complexity of the app?

I know of Python's GIL, and I assume Django creators would've obviously designed processes to avoid GIL-related issues. This answer speaks a bit about how multiple web requests are handled, and I've come to understand that I have to use external libraries which would use WSGI to perform load balancing. So I'm assuming the load balancer would run multiple instances of my webapp in separate processes to serve client requests, and I'd only have to worry about avoiding race conditions and mutual exclusion issues.

 

In most cases that solves the problem and is easy to setup. I suggest you do it that way (I have used Gunicron, uwsgi and mod_wsgi depending on what the client wanted or needed) and look at more elaborate solutions when you need to - it might be never.

I also searched for benchmarks, and found some Locust tests which seem like Django could handle around 500 requests per second without much sweat. The reason I need to know this, is because I need to know what the threshold is (for number of simultaneous user connections), before I consider installing something as complex as Kubernetes. How many requests can be handled by an ordinary load balancer before resorting to Kubernetes?

How were you planning to use Kubernates to balance load? Unless I am missing something it would not let you handle any more, just make it easier to deploy.