Django Channels: best way to launch multiple workers

[Charlie DeTar]

What's the best way to launch multiple Django channels workers on a production server to take advantage of extra cores?

The documentation says:

Each server is single-threaded, so it’s recommended you run around one or two per core on each machine; it’s safe to run as many concurrent workers on the same machine as you like, as they don’t open any ports (all they do is talk to the channel backend).

However, `python manage.py help runworker` lists a "--threads" option, which seems to imply that a single invocation of runworker can launch multiple workers.

Is there a functional difference between `./manage.py runworker --threads 4` and launching `./manage.py runworker` process 4 times?

best,

Charlie

[Andrew Godwin]

The difference is mostly in Python performance - threading in Python tends to perform worse than using multiple processes, which is why we recommend using multiple processes in the docs. However, you can save a bit of memory usage with threading, so you can use that if you want.

I would not, however, recommend running more than 2 - 4 threads per process, as otherwise you'll likely see performance take a slide due to the GIL.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8097b8f7-3f89-49a6-b189-58a0713972ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Piet van Leeuwen]

On a server is there a recommendation for how many workers to have running?

With Gunicorn we would run...

def max_workers():
    return cpu_count() * 2 + 1

On Thursday, February 9, 2017 at 11:38:07 AM UTC+13, Andrew Godwin wrote:

The difference is mostly in Python performance - threading in Python tends to perform worse than using multiple processes, which is why we recommend using multiple processes in the docs. However, you can save a bit of memory usage with threading, so you can use that if you want.

I would not, however, recommend running more than 2 - 4 threads per process, as otherwise you'll likely see performance take a slide due to the GIL.

Andrew

On Wed, Feb 8, 2017 at 11:53 AM, Charlie DeTar <cha...@gmail.com> wrote:

What's the best way to launch multiple Django channels workers on a production server to take advantage of extra cores?

The documentation says:

Each server is single-threaded, so it’s recommended you run around one or two per core on each machine; it’s safe to run as many concurrent workers on the same machine as you like, as they don’t open any ports (all they do is talk to the channel backend).

However, `python manage.py help runworker` lists a "--threads" option, which seems to imply that a single invocation of runworker can launch multiple workers.

Is there a functional difference between `./manage.py runworker --threads 4` and launching `./manage.py runworker` process 4 times?

best,

Charlie

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

[Andrew Godwin]

I don't have enough hard data to give a recommendation, unfortunately.

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/96819d5d-cb83-4e11-af79-3c69b16f8b3b%40googlegroups.com.

[Duane Hilton]

Is there any update on this? Do we have any recommendations now on how many workers can run on a server?

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/96819d5d-cb83-4e11-af79-3c69b16f8b3b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFwN1upDm3iJDCgOWMqkB8od8NKCpBhhH73GqvrjsYTRn3ow9g%40mail.gmail.com.