maximum number of connection using django-channels

[Gopal]

Hello everyone,
I am using django=1.10.5 and channels==1.0.3 in my project.
First i will give you little description about my project so that you all can understand my question properly.
I am getting latitude and longitude of android device(first user) using websockes. So that android application is connected to my django server(second user) through websocket (implemented using channels), and i am storing all latitude of longitude of all connected android devices using that app. After that i am sending all those locations to web browsers(third user). So basically i am taking location from android devices (or from first user), storing into server's database (or to second user) and showing those location to web browser (or to third user). They all are communicating using websocket. Now my question is how many clients or user can connect to that websocket and does number of connections depend upon free ports or does it depend on RAM and second question is that can some one explain in detail from OS perspective that how does django-channel works ???

[Andrew Godwin]

Hi,

There's more about how Channels works in the docs and in talks I've given, but the short version is:

* Websockets go into a server called Daphne, which is written in Twisted and so can handle hundreds or potentially thousands of simultaneous connections open at once

* Any event on a websocket (connect, message received) is sent onto the channel layer, which is a type of first-in-first-out queue

* Django worker processes synchronously take a message from the queue, process it, and then loop back to take another

Thus, there are two factors to scaling:

* The number of open connections affects how many Daphne instances you run

* The throughput of events affects the number of workers you run

For example, if you had 10000 clients connecting but sending one message every few minutes, you'd want 10 Daphne instances but only a few workers. If you had 100 clients sending 10 messages a second, you'd only need one Daphne instances but lots of workers.

The number of connections Daphne can take depends on a combination of CPU speed and kernel limits; it's not going to be RAM-limited except on very low memory systems.

Hope that helps.

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a9cd7ffd-0f69-4aff-b6c2-8ae4bf9df091%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Carrell]

Can I please ask a clarifying question?

You examples above ("if you had 10000 clients...") spoke to clients sending the messages. I have a use case where the clients are never (at the moment) sending messages but rather listening for messages from the server. When the server has something to say it does so via Groups. Each client will be subscribed to 3 different Consumers. It is likely that there will be as many as 30 - 60 total clients each on a Group with no more than a few (10s at the very most extreme) other reply_channels.

Can you comment on the limitations of channels in this scenario?

Fair warning: We are currently running Django as a development server in "production" (It's an internal tool.) so we're not manually using Daphne and a standalone server and WSGI, etc. It's just a single instance of "runserver". I'm working on convincing our team that we should change our deployment model from its initial prototype but there is not a lot of perceived value in that effort at the moment.

Thanks!

On Thursday, March 2, 2017 at 11:26:18 AM UTC-6, Andrew Godwin wrote:

Hi,

There's more about how Channels works in the docs and in talks I've given, but the short version is:

* Websockets go into a server called Daphne, which is written in Twisted and so can handle hundreds or potentially thousands of simultaneous connections open at once

* Any event on a websocket (connect, message received) is sent onto the channel layer, which is a type of first-in-first-out queue

* Django worker processes synchronously take a message from the queue, process it, and then loop back to take another

Thus, there are two factors to scaling:

* The number of open connections affects how many Daphne instances you run

* The throughput of events affects the number of workers you run

For example, if you had 10000 clients connecting but sending one message every few minutes, you'd want 10 Daphne instances but only a few workers. If you had 100 clients sending 10 messages a second, you'd only need one Daphne instances but lots of workers.

The number of connections Daphne can take depends on a combination of CPU speed and kernel limits; it's not going to be RAM-limited except on very low memory systems.

Hope that helps.

Andrew

On Wed, Mar 1, 2017 at 10:01 PM, Gopal <gopalm...@gmail.com> wrote:

Hello everyone,
I am using django=1.10.5 and channels==1.0.3 in my project.
First i will give you little description about my project so that you all can understand my question properly.
I am getting latitude and longitude of android device(first user) using websockes. So that android application is connected to my django server(second user) through websocket (implemented using channels), and i am storing all latitude of longitude of all connected android devices using that app. After that i am sending all those locations to web browsers(third user). So basically i am taking location from android devices (or from first user), storing into server's database (or to second user) and showing those location to web browser (or to third user). They all are communicating using websocket. Now my question is how many clients or user can connect to that websocket and does number of connections depend upon free ports or does it depend on RAM and second question is that can some one explain in detail from OS perspective that how does django-channel works ???

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

[Andrew Godwin]

Hi John,

It's impossible to give any accurate estimate of performance numbers as it depends so heavily on hardware. Instead, I can point out to you likely bottlenecks:

- Each instance of Daphne can only terminate so many connections. I would hope this is above 1000 on any reasonable machine.

- Each instance can only handle so many messages in or out per second. This is a function of how many connections you have combined with how chatty they are.

- Sending to groups currently scales linearly with group size, so large groups are slower to send to

Ultimately, you'll have to do your own performance measurements with your environment and your code to work out what you can do. I would say, however, don't run "runserver" in production!

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7351d523-e636-49aa-b7dd-0d28db976a55%40googlegroups.com.