Django Channels Redis: MaxClientsError

[Cédric Foellmi]

Hello everyone. I am seeking some help with Django Channels and Redis.

I have a Django app running on Heroku. This app is rather simple: a Daphne server, a background worker and a scheduling beat worker (the last two, using Celery).

I noticed I hit *very often* Redi's MaxClientsError (every 15 minutes or so) despite using the non-free plan of RedisCloud on Heroku, which has the following parameters: Memory Size = 100 MB, Dedicated DB = 4, Connections = 256. I was using the free plan for a while with 30 connections, but the MaxClientsError was popping all the time. Thus I increased to 256 connections, but since I have more bakground tasks, I hit again the error.

In the stacktrace below, you can see the problem occurs within djangochannelsrestframework code. Line 233, there is this:

for group_name in group_names:
	            async_to_sync(channel_layer.group_send)(group_name, message)

But we never managed to understand if that was the cause of the error.  Is that way of doing the reason for opening so many connections ? My app is not a chat. It has only background tasks that discover some data remotely, and send them through websockets for a kind of "live events RSS".

Wouldn't it be better to use something like:

async2sync = async_to_sync(channel_layer.group_send)

for group_name in group_names:

async2sync(group_name, message)

Thanks a lot in advance!!!

---

My conf is the following:

Django==2.2.*

channels==2.2.*

channels-redis==2.3.*

daphne==2.3.*

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels_redis.core.RedisChannelLayer",
        "CONFIG": {
            "hosts": [os.environ.get('REDISCLOUD_URL', 'redis://localhost:6379')],
        },
        "symmetric_encryption_keys": [SECRET_KEY],
    },
}


And the stacktrace: 

MaxClientsError: ERR max number of clients reached
  File "arcsecond/activities/tasks/archive_datarows.py", line 65, in parse_archive_latest_rows
    target_name=row.target_name)
  File "django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "django/db/models/base.py", line 741, in save
    force_update=force_update, update_fields=update_fields)
  File "django/db/models/base.py", line 790, in save_base
    update_fields=update_fields, raw=raw, using=using,
  File "django/dispatch/dispatcher.py", line 175, in send
    for receiver in self._live_receivers(sender)
  File "django/dispatch/dispatcher.py", line 175, in <listcomp>
    for receiver in self._live_receivers(sender)
  File "djangochannelsrestframework/observer/observer.py", line 154, in post_save_receiver
    **kwargs
  File "djangochannelsrestframework/observer/observer.py", line 221, in post_change_receiver
    **kwargs
  File "djangochannelsrestframework/observer/observer.py", line 233, in send_messages
    async_to_sync(channel_layer.group_send)(group_name, message)
  File "asgiref/sync.py", line 79, in __call__
    return call_result.result()
  File "concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "asgiref/sync.py", line 95, in main_wrap
    result = await self.awaitable(*args, **kwargs)
  File "channels_redis/core.py", line 559, in group_send
    async with self.connection(self.consistent_hash(group)) as connection:
  File "channels_redis/core.py", line 742, in __aenter__
    self.conn = await self.pool.pop()
  File "channels_redis/core.py", line 49, in pop
    conns.append(await aioredis.create_redis(**self.host, loop=loop))
  File "aioredis/commands/__init__.py", line 178, in create_redis
    loop=loop)
  File "aioredis/connection.py", line 129, in create_connection
    await conn.auth(password)
  File "aioredis/util.py", line 48, in wait_ok
    res = await fut

[Sabuhi Shukurov]

hello, have you solved this issue?