[Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
Django
timeouts difficult
------------------------------------------------+------------------------
Reporter: bpicolo | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Core (Cache system) | Version: 4.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
Hello! First Django interaction, so please let me know how better I can
give info here.
I discovered unintentionally that there's no default socket_timeout set
for Redis cache connections. This is an issue on it's own (unsafe default
that's a particularly hard issue to track down), but in trying to set
those timeouts, I hit some tough edges of documentation/usage.
The cache documentation says that, for caches backed by third-party
libraries, the OPTIONS object
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library
is passed to the underlying library connection object]. This is reaffirmed
for the redis cache specifically
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors
here].
Right now, though, these OPTIONS are passed to the
[https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30
first-party RedisCacheClient object], so you can't pass in options
expected by the redis connection pool [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1283 link].
One could do a variety of subclassing to make this work, but there's quite
a few layers of indirection here, so it's hard to identify exactly what
one should subclass in order to get objects to the connection class
appropriately.
(For now, my workaround is take advantage of the from_url behavior where
the connection class [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1274 pulls arguments out of the
connection query string], but this isn't straightforward and feels
brittle).
There are a couple of options here – revising documentation or changing
behavior (or both).
In my ideal case, it would be great for it to be easy to pass down
explicit socket and other timeouts. To me, that suggests updating behavior
is ideal, because the documentation would potentially lead folk down a
path of tough to navigate behavior.
--
Ticket URL: <https://code.djangoproject.com/ticket/33681>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
timeouts difficult
Reporter: bpicolo | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Cache system) | Version: 4.0
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by bpicolo:
Old description:
New description:
Hello! First Django interaction, so please let me know how better I can
give info here.
I discovered unintentionally that there's no default socket_timeout set
for Redis cache connections. This is an issue on it's own (unsafe default
that's a particularly hard issue to track down), but in trying to set
those timeouts, I hit some tough edges of documentation/usage.
The cache documentation says that, for caches backed by third-party
libraries, the OPTIONS object
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library
is passed to the underlying library connection object]. This is reaffirmed
for the redis cache specifically
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors
here].
Right now, though, these OPTIONS are passed to the
[https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30
first-party RedisCacheClient object], so you can't pass in options
expected by the redis connection pool [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1283 link].
One could do a variety of subclassing to make this work, but there's quite
a few layers of indirection here, so it's hard to identify exactly what
one should subclass in order to get arguments to the connection class
appropriately.
(For now, my workaround is take advantage of the from_url behavior where
the connection class [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1274 pulls arguments out of the
connection query string], but this isn't straightforward and feels
brittle).
There are a couple of options here – revising documentation or changing
behavior (or both).
In my ideal case, it would be great for it to be easy to pass down
explicit socket and other timeouts. To me, that suggests updating behavior
is ideal, because the documentation would potentially lead folk down a
path of tough to navigate behavior.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33681#comment:1>
Django
Old description:
> what one should subclass in order to get arguments to the connection
> class appropriately.
>
> (For now, my workaround is take advantage of the from_url behavior where
> the connection class [https://github.com/redis/redis-
> py/blob/master/redis/connection.py#L1274 pulls arguments out of the
> connection query string], but this isn't straightforward and feels
> brittle).
>
> There are a couple of options here – revising documentation or changing
> behavior (or both).
>
> In my ideal case, it would be great for it to be easy to pass down
> explicit socket and other timeouts. To me, that suggests updating
> behavior is ideal, because the documentation would potentially lead folk
> down a path of tough to navigate behavior.
New description:
Hello! First Django interaction, so please let me know how better I can
give info here.
I discovered unintentionally that there's no default socket_timeout set
for Redis cache connections. This is an issue on it's own (unsafe default
that's a particularly hard issue to track down), but in trying to set
those timeouts, I hit some tough edges of documentation/usage.
The cache documentation says that the OPTIONS object is passed to the
third-party connection class for connections backed by third-party
libraries
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library
is passed to the underlying library connection object]. This is reaffirmed
for the redis cache specifically
[https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors
here].
Right now, though, these OPTIONS are passed to the
[https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30
first-party RedisCacheClient object], so you can't pass in options
expected by the redis connection pool [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1283 link]. In that sense,
RedisCacheClient is being treated as a "first-party" cache according to
these docs, but it defers directly to the redis library.
One could do a variety of subclassing to make this work, but there's quite
a few layers of indirection here, so it's hard to identify exactly what
one should subclass in order to get arguments to the connection class
appropriately.
(For now, my workaround is take advantage of the from_url behavior where
the connection class [https://github.com/redis/redis-
py/blob/master/redis/connection.py#L1274 pulls arguments out of the
connection query string], but this isn't straightforward and feels
brittle).
There are a couple of options here – revising documentation or changing
behavior (or both).
In my ideal case, it would be great for it to be easy to pass down
explicit socket and other timeouts. To me, that suggests updating behavior
is ideal, because the documentation would potentially lead folk down a
path of tough to navigate behavior.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33681#comment:2>
Django
timeouts difficult
-------------------------------------+-------------------------------------
Reporter: bpicolo | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Cache system) | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by bpicolo:
Old description:
> Hello! First Django interaction, so please let me know how better I can
> give info here.
>
> I discovered unintentionally that there's no default socket_timeout set
> for Redis cache connections. This is an issue on it's own (unsafe default
> that's a particularly hard issue to track down), but in trying to set
> those timeouts, I hit some tough edges of documentation/usage.
>
New description:
for connections backed by third-party libraries]. This is reaffirmed for
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33681#comment:3>
Django
Old description:
New description:
to django implementation details).
There are a couple of options here – revising documentation or changing
behavior (or both).
In my ideal case, it would be great for it to be easy to pass down
explicit socket and other timeouts. To me, that suggests updating behavior
is ideal, because the documentation would potentially lead folk down a
path of tough to navigate behavior.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33681#comment:4>
Django
Old description:
New description:
path of tough to navigate behavior. Passing those options in tact, or
updating the RedisCacheClient to have a wider range of available options
would both work out.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33681#comment:5>