[Channels] Recommended quantity of workers

[Renato Oliveira]

Hi

Is there any recommendation on how many `runworker` processes should we run on production?

I'm having many 403 responses on frontend and ChannelFull on backend. Saw here on the mailing list that adding new workers might help. Indeed it helped, tested already with 10 for now, but would like to know if there is any side effect doing this.

Thanks!

 

Renato Oliveira

Co-founder & Software Engineer

+55 81 987855061

www.labcodes.com.br

[Andrew Godwin]

The number you need depends on the kind of site you have and the type of traffic - much like "how many WSGI servers should I run?" there is no right answer. You'll need to look at how different numbers of workers affect your median response times and number of timeouts and adjust appropriately.

There is no downside to running multiple workers, though; in fact, it's encouraged.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAGPLouWycQAfpJDAimYbxTUcoZkNMx3RGDgGf3FheHSyZYQ6yQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Renato Oliveira]

Thanks Andrews!

Also thanks for your hard work on channels! Awesome piece of software.

 

Renato Oliveira

Co-founder & Software Engineer

+55 81 987855061

www.labcodes.com.br

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFwN1up6_56kxBFJ-p3b2Enz5nP-MiDxsQB0%2BeNi9phrNsmrbA%40mail.gmail.com.

[Renato Oliveira]

Andrew*

 

Renato Oliveira

Co-founder & Software Engineer

+55 81 987855061

www.labcodes.com.br

[James Schneider]

Is there any recommendation on how many `runworker` processes should we run on production?

There is no real guideline to provide. The "correct" number is a function of hardware, traffic, and the amount of processing needed to fulfill requests. That number may change over time as the app is further developed.

Increase the number of available processes/threads across some amount of time (perhaps a few days or a week to account for bursts). Be sure to monitor CPU, RAM, and disk I/O. If any of those reach 100% utilization during regular operation (or your response times drop), back it off slightly to account for bursts. Keep in mind that you may also need to tune the number of database listeners or connections, and/or RAM allocated to the DB cache, etc. Other dependent pieces of your infrastructure may need modification as well, such as caches.

I'm having many 403 responses on frontend and ChannelFull on backend. Saw here on the mailing list that adding new workers might help. Indeed it helped, tested already with 10 for now, but would like to know if there is any side effect doing this.

No bad side effects until you exceed RAM, CPU, or disk limits. In many cases, your DB is the bottleneck, so be sure to tune those settings along with the web service processes.

It's a bit strange to receive a 403, since that generally has to do with authorization, and not operational issues. Seems like a 5XX error would be more appropriate for a ChannelFull situation, but that may be just the way it works.

-James