Django channels scalling

[Divit Rao]

Hi Team,

currently I have deployed a Django project on EC2 server which has a RAM of about 2GB using Gunicorn and Ngnix, I'm using Django channels on same server, so as of now we have around 200 active users. currently each user will have 5 active WebSocket connections/consumers (for chat, notification, etc.) open so currently around 1000 open connections are present at a time. As of now WebSocket's are working fine but in future we are expecting a growth in user so as the active WebSocket connections will grow, and at some point, due to excessive load, the server will start throwing issues regarding socket connections my question is how to scale django channels.

Like should i just increase server size or is there any other solutions? 

[Shaheed Haque]

Generally, as you may know, it is not very easy to get memory usage stats on Unix/Linux without specialised tooling.

However, in my experience, with default TCP settings, the kernel overhead plus whatever "typical" app overhead will not often exceed say 100 kB per connection.

Now, we had originally used an even smaller 1GB server than yours, and we did run out of memory, with attendant catastrophic failures. We were able to trace the failures to certain ORM queries on tables with large JSONB columns (in the MB range). Once we addressed that, we were fine. At no point, even with our usage  pattern of connect per message (!!!) , did sockets usage even warrant a second thought. 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/3e759081-5b3e-4a74-9668-e54e2e4e9d56n%40googlegroups.com.

[Divit Rao]

Thanks mate 

[Angel Galicia Sp]

From my point of view, You should avoid multi ws connections per client, the purpose of a ws connection is to be able to send data when something happens without a new user request, that being said... I think that the way DJ Channels is built makes it difficult to have just one connection per client since for each path in your router you have 1 consumer and for each consumer you have a  websocket_connect method which creates a new ws connection so at first sight, is not clear how to avoid this behavior and some times is just not possible (At least not without too much effort). I have several apps with django channels with many customizations in order to reuse the main ws connection and it works but I'm not completely sure about the right way to achieve this (one ws connection per client). So, if someone has more experience with it, please share.