[Django] #33573: Add native async support to redis cache backend

29 views
Skip to first unread message

Django

unread,
Mar 11, 2022, 11:47:46 AM3/11/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-----------------------------------------------+------------------------
Reporter: AngellusMortis | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------------+------------------------
The latest version of redis-py will support both sync and async clients so
`sync_to_async` will no longer be necessary so it would be nice if the out
of the box Redis cache backend supports both natively as well.

https://github.com/redis/redis-py/releases/tag/v4.2.0rc1

It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
ticket out there so it can be on someones radar.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Mar 14, 2022, 4:14:50 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+------------------------------------
Reporter: Christopher Bailey | Owner: nobody

Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by Mariusz Felisiak):

* cc: Andrew Chen Wang, Andrew Godwin, Carlton Gibson (added)
* stage: Unreviewed => Accepted


Comment:

Tentatively accepted. Do you have an idea for implementation? As far as
I'm aware this would require initializing two redis clients: async and
non-async.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:1>

Django

unread,
Mar 14, 2022, 4:25:16 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------

Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* stage: Accepted => Someday/Maybe


Comment:

Let's wait for the final release first.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:2>

Django

unread,
Mar 14, 2022, 9:33:48 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Andrew Chen Wang):

We are currently re-implementing the async client (one last time) in a way
that allows for aioredis 1.3.1 performance.
[https://github.com/seandstewart/redis-py-
sansio/blob/main/benchmark/conftest.py] The RedisLab team will need to
approve it first, so I would wait until our new client is either
approved/denied [https://github.com/redis/redis-py/issues/2039]. This
won't affect the non-async client as the non-async client will still
default on the current redis-py interface with the option to also use the
new API.

This isn't to say no one should try implementing the async client now; I
have high confidence the new client will be merged in with Redis-py as
we're gunning for the much-needed performance gain. Just don't be
surprised if a few things shift around.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:3>

Django

unread,
Mar 14, 2022, 10:22:13 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Christopher Bailey:

Old description:

> The latest version of redis-py will support both sync and async clients
> so `sync_to_async` will no longer be necessary so it would be nice if the
> out of the box Redis cache backend supports both natively as well.
>
> https://github.com/redis/redis-py/releases/tag/v4.2.0rc1
>
> It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
> ticket out there so it can be on someones radar.

New description:

Do you have an idea for implementation? As far as I'm aware this would
require initializing two redis clients: async and non-async.

I kind of started to implement it, but then realized it was likely going
to be a bigger issue then I though. But there is already an abstract
RedisClient class that is a wrapper around the base redis.Redis class. It
maintains connection polls and handles all of the core commands. It has
methods for `get`, `set`, `add`, `touch`, `incr` that just calls
`get_client` and makes a new client using the existing connection pull any
time it is called.

For Async we can just add something like `get_async_client` and follow the
same pattern within the class. I believe the connection poll should be
able to be reused between the sync/async clients.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:4>

Django

unread,
Mar 14, 2022, 10:24:34 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Old description:

> Do you have an idea for implementation? As far as I'm aware this would
> require initializing two redis clients: async and non-async.
>
> I kind of started to implement it, but then realized it was likely going
> to be a bigger issue then I though. But there is already an abstract
> RedisClient class that is a wrapper around the base redis.Redis class. It
> maintains connection polls and handles all of the core commands. It has
> methods for `get`, `set`, `add`, `touch`, `incr` that just calls
> `get_client` and makes a new client using the existing connection pull
> any time it is called.
>
> For Async we can just add something like `get_async_client` and follow
> the same pattern within the class. I believe the connection poll should
> be able to be reused between the sync/async clients.

New description:

The latest version of redis-py will support both sync and async clients so
`sync_to_async` will no longer be necessary so it would be nice if the out
of the box Redis cache backend supports both natively as well.

https://github.com/redis/redis-py/releases/tag/v4.2.0rc1

It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
ticket out there so it can be on someones radar.

--

Comment (by Christopher Bailey):

Do you have an idea for implementation? As far as I'm aware this would
require initializing two redis clients: async and non-async.

I kind of started to implement it, but then realized it was likely going
to be a bigger issue then I though. But there is already an abstract
RedisClient class that is a wrapper around the base redis.Redis class. It
maintains connection polls and handles all of the core commands. It has
methods for `get`, `set`, `add`, `touch`, `incr` that just calls
`get_client` and makes a new client using the existing connection pull any
time it is called.

For Async we can just add something like `get_async_client` and follow the
same pattern within the class. I believe the connection poll should be
able to be reused between the sync/async clients.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:5>

Django

unread,
Mar 14, 2022, 10:24:41 AM3/14/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Christopher Bailey):

Do you have an idea for implementation? As far as I'm aware this would
require initializing two redis clients: async and non-async.

I kind of started to implement it, but then realized it was likely going
to be a bigger issue then I though. But there is already an abstract
RedisClient class that is a wrapper around the base redis.Redis class. It
maintains connection polls and handles all of the core commands. It has
methods for `get`, `set`, `add`, `touch`, `incr` that just calls
`get_client` and makes a new client using the existing connection pull any
time it is called.

For Async we can just add something like `get_async_client` and follow the
same pattern within the class. I believe the connection poll should be
able to be reused between the sync/async clients.

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:6>

Django

unread,
Mar 23, 2022, 12:49:37 PM3/23/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Christopher Bailey):

4.2.0 is fully out now.

Django

unread,
Dec 31, 2022, 1:38:54 PM12/31/22
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Andrew Chen Wang):

It doesn't seem like the Sans-IO approach will be adopted anytime soon as
work and review has stalled.

> I believe the connection poll should be able to be reused between the

sync/async clients, but I am not an expert on Redis.

Sorry if I misunderstood, but the ConnectionPool classes for the sync and
async versions are different, and the Connection classes are also
different.

> As far as I'm aware this would require initializing two redis clients:
async and non-async.

Assuming you mean initialize `redis.Redis` and `redis.asyncio.Redis` in
one class, I disagree; I think we should have two RedisCache classes, one
for sync and another for async.

1. Some users may want both a sync and async Redis client. However, the
options presented to devs cater to only one of the variants. For instance,
if you need to pass in a custom ConnectionPool or provide pool options for
the default pools, those could be different for the different environments
like an async enabled middleware uses the async Redis but a sync view uses
the sync Redis. If a custom Connection class is provided to the pool, then
we'd have a problem since the Connection class for the sync and async
Redis are different.
2. The actual network calls like `get` and `set_many` look like they will
be copy-paste because of the numerous awaits required, so there won't be
much if any re-use of code
3. Not all clients have upgraded to at least Redis 4.2.0.
4. It sets a paradigm for future client implementations. Regardless of
whether packages like psycopg(3) implement both a sync and async variant
or just one type like the memcached packages implementing just sync, there
should be a separation of clients based on environment to set future
expectations of clients not necessarily having a counterpart variant like
an async package not having sync support.

Let me know your thoughts. Thanks!

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:7>

Django

unread,
Jan 3, 2023, 1:17:44 AM1/3/23
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Carlton Gibson):

Hey Andrew, thanks for the follow-up!

If I've understood you correctly, I think the best way forward would be to
prototype the async backend in a external module, so we can see what it
would look like and install it and give it a run. I think we'd need to
have a some discussion as to whether/how we'd adopt that then into Django,
and the messaging we'd put around that.

Make sense?

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:8>

Django

unread,
Jan 10, 2023, 4:18:57 PM1/10/23
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Andrew Chen Wang):

Makes sense. Thanks for the comment!

You can take a look at a practically broken but conceptual implementation
here: https://github.com/Andrew-Chen-Wang/django-async-redis/pull/6

The repository itself is designed to be a django-redis replacement that I
recently migrated from aioredis to redis-py; in regards to this convo, the
places to look are `django_async_redis.client.core.py` and the test file
`tests/test_core.py`

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:9>

Django

unread,
Mar 18, 2024, 3:08:36 AM3/18/24
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: nobody
Type: New feature | Status: new
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Ülgen Sarıkavak):

* cc: Ülgen Sarıkavak (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:10>

Django

unread,
Oct 12, 2024, 4:03:26 AM10/12/24
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Ahmed Ibrahim):

* owner: nobody => Ahmed Ibrahim
* status: new => assigned

Comment:

I can take this over, any ideas on what's the final solution? I will
propose a solution soon as well after investigation
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:11>

Django

unread,
Oct 12, 2024, 5:58:43 PM10/12/24
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Andrew Chen Wang):

@Ahmed I'm forgetting context since it's been awhile, but my draft seems
to have some ideas: https://github.com/Andrew-Chen-Wang/django-async-
redis/pull/6. Specifically:
1. A lot of the code seems to be copying the sync to async prefixed
classes and methods
2. There are some packages that use the default cache client. It needs to
know whether the client is async/sync capable, so you may need to add some
variable (similar to how middleware classes are async/sync capable via a
variable). The reason I made two classes, one for sync and another for
async, is so that the client caller can know whether to execute as sync or
async. I believe work stalled on my end trying to determine how to resolve
the what context the client caller is coming from.
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:12>

Django

unread,
Oct 12, 2024, 6:55:02 PM10/12/24
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Christopher Bailey):

I do not think have two separate clients is a good approach. It is
actually a really bad one for apps that have a lot of caches. If you have
a legacy WSGI app you want to migrate to ASGI and you have 20 caches (not
as crazy as it sounds for larger apps with very high throughput). Now you
have to duplicate all of them, one for sync and one for async, and have 40
caches?

If a cache is capable of both sync and async, it should raise an exception
when used in the wrong context rather than forcing it to be a
configuration/usage issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:13>

Django

unread,
Jan 2, 2025, 4:09:10 AMJan 2
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce):

Note that #36047 was raised while working with an async cache client
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:14>

Django

unread,
Jan 3, 2025, 6:11:27 AMJan 3
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by amirreza):

* cc: amirreza (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:15>

Django

unread,
Jan 3, 2025, 6:13:50 AMJan 3
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by amirreza):

hi, i'd like to add that the above issue is not redis exclusive, and
happens with any cache backend, if async is used
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:16>

Django

unread,
Jan 3, 2025, 6:19:26 AMJan 3
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by amirreza):

Replying to [comment:13 Christopher Bailey]:
> I do not think have two separate clients is a good approach. It is
actually a really bad one for apps that have a lot of caches. If you have
a legacy WSGI app you want to migrate to ASGI and you have 20 caches (not
as crazy as it sounds for larger apps with very high throughput). Now you
have to duplicate all of them, one for sync and one for async, and have 40
caches?

i'm not sure why we would keep the sync version when the async one is
added, but merging both sync and async into one client is not easy, unless
asgiref is used.
cache clients such as `redis-py` have different clients for sync and
async, combining them into one class needs hard work to implement, and to
maintain.
also, what if i want my client to be async/sync only?

for your issue i'd suggest a wrapper around sync and async clients, so
both clients exists, and duplication won't happen.
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:17>

Django

unread,
Jan 26, 2025, 6:57:52 AMJan 26
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by amirreza):

hi
i have a feeling this is abandoned, but if not, i'd like to take over

the way i will do it is by using seprate classes for sync and async, using
redis's native support for async, as i explained in my previous message

this way the issue i riased at #36047 will also be easy to fix (i think)
i'll tackle #36047 afterwards
the way i'll do is to check if the backend has an async `aclose` method,
if so, call `async_to_sync()` on it
tho this one is a raw idea and not totaly sure about it

let me know if this is ok with you
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:18>

Django

unread,
Apr 7, 2025, 9:51:54 PMApr 7
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by amirreza):

hi 🙌🏼
so after a good amount of time to investigate things, i'm back to talk
more on the subject
in this commecnt i'll try to share what an async client would look like in
django, and leave it to you to decide if this is desired or not


== so the first problem that needs solving is the
[https://github.com/django/django/blob/main/django/core/cache/__init__.py#L61-L64)
cleaup] discussed in #36047
since this is a signal receiver and signal receiver can be
[https://docs.djangoproject.com/en/5.1/topics/signals/#receiver-functions
async] since django 5, i can see this fixed like this:

1. add a `is_async` attribute to cache clients, similar to
[https://github.com/django/django/blob/main/django/utils/deprecation.py#L87-L88
middlewares]

2. change the receiver to be async, like this:
{{{
async def close_caches(**kwargs):
# Some caches need to do a cleanup at the end of a request cycle. If
not
# implemented in a particular backend cache.close() is a no-op.
await caches.close_all()
}}}
where `close_all()` is as follows:
{{{
async def close_all(self):
for conn in self.all(initialized_only=True):
if conn.is_async:
await conn.aclose()
else:
conn.close()
}}}



== second matter in hand is how to design the async cache backend
i personally suggest that async backend should be in a different class, i
hope by the end of this comment you'd agree too

so i would do something like this:

1. a base class with common functionality
2. two (less)base classes for sync and async functionality
3. two redis backends for sync and async

async methods would have `a` prefixed to their name (even tho there are no
sync equivelents)
for two reason:

1. to be compatible with rest of django
2. so if someone wants (for whatever reason), they can combine the two
class and have a backend that supports both

tho if you allow me to be naughty a bit🫣, i'll define a `__getattr__`
that only works in the async backend and allows people to access methods
without the `a`
i've planned a similar thing with [https://github.com/amirreza8002/django-
valkey/blob/commands/django_valkey/base.py#L342-L349 django-valkey]
so both `await cache.aset()` and `await cache.set()` work





== some things to have in mind:
❕the following problems are not essential part of this ticket, an async
cache backend can exists without these
even a 3rd-party can handle these issues, i'm only mentioning for refrence

1. cache middlewares do not accept an async only cache backend
to solve this we either have to combine the two backends (as discussed
above) (in this case the async methods won't even be called, it just
doesn't break),
or have new middlewares that support async (i like this one, but i guess
others don't)


2. if we decide to make an async middleware,
[https://github.com/django/django/blob/main/django/utils/decorators.py#L124
make_middleware_decorator] might not be very happy, since it expects sync
methods, which don't make sense in an async middleware
i have made an async only option in a [https://github.com/django-utils
/django-async-
extensions/blob/main/django_async_extensions/utils/decorators.py#L31 3rd-
party] before


3. again if we move for an async middleware,
[https://github.com/django/django/blob/main/django/views/decorators/cache.py#L10,
cache_page] decorator won't work, needs another one


4. these two caching utils expect a sync backend
[https://github.com/django/django/blob/main/django/utils/cache.py#L377]
and
[https://github.com/django/django/blob/main/django/utils/cache.py#L399]


5. if cache server is used for sessions, the session middleware doesn't
support async operations
the solutions mentionedfor cache middleware also applies here


i hope this can give some calrity on how this would work
as before, i can implement this, if this is a feature that people want,
and if not, well at least this can be used as refrence for other people

with best of wishes♥️
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:19>

Django

unread,
Jul 17, 2025, 1:43:54 AMJul 17
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Ahmed Ibrahim):

Replying to [comment:18 amirreza]:
> hi
> i have a feeling this is abandoned, but if not, i'd like to take over
>
> the way i will do it is by using seprate classes for sync and async,
using redis's native support for async, as i explained in my previous
message
>
> this way the issue i riased at #36047 will also be easy to fix (i think)
> i'll tackle #36047 afterwards
> the way i'll do is to check if the backend has an async `aclose` method,
if so, call `async_to_sync()` on it
> tho this one is a raw idea and not totaly sure about it
>
> let me know if this is ok with you


dw I will do it, but I need a Django fellow to provide a certain answer on
which implementation we should go with, since we can do it and it won't
get accepted, I "think" (not sure) I emailed before in the mailing list,
but got no response, I'm not sure if that's deja vu or something
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:20>

Django

unread,
Jul 18, 2025, 10:20:16 AMJul 18
to django-...@googlegroups.com
#33573: Add native async support to redis cache backend
-------------------------------------+-------------------------------------
Reporter: Christopher Bailey | Owner: Ahmed
| Ibrahim
Type: New feature | Status: assigned
Component: Core (Cache system) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by amirreza):

Replying to [comment:20 Ahmed Ibrahim]:
> Replying to [comment:18 amirreza]:
> > hi
> > i have a feeling this is abandoned, but if not, i'd like to take over
> >
> > the way i will do it is by using seprate classes for sync and async,
using redis's native support for async, as i explained in my previous
message
> >
> > this way the issue i riased at #36047 will also be easy to fix (i
think)
> > i'll tackle #36047 afterwards
> > the way i'll do is to check if the backend has an async `aclose`
method, if so, call `async_to_sync()` on it
> > tho this one is a raw idea and not totaly sure about it
> >
> > let me know if this is ok with you
>
>
> dw I will do it, but I need a Django fellow to provide a certain answer
on which implementation we should go with, since we can do it and it won't
get accepted, I "think" (not sure) I emailed before in the mailing list,
but got no response, I'm not sure if that's deja vu or something


hi
we discussed it in https://forum.djangoproject.com/t/need-input-for-async-
redis-backend/38426/8
i've also provided some info about some of the problems async caching has,
that need solving.

i think the current idea is to do 3rd party work.
as i mentioned in the forum, i have a pr in `django-valkey` that would do
the work
i'm currently examining it, i'd be glad if i get a review
--
Ticket URL: <https://code.djangoproject.com/ticket/33573#comment:21>
Reply all
Reply to author
Forward
0 new messages