[Django] #31938: page cache invalidation
Django
------------------------------------------+------------------------
Reporter: Laurent Tramoy | Owner: nobody
Type: New feature | Status: new
Component: Utilities | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------+------------------------
I've recently started to use caching for some of my pages, through the
`cache_page` decorator, everything works fine.
However, as soon as any dev implements a caching mechanism, they also need
to think about cache invalidation, and that's my problem: there is no easy
way to invalidate a specific page in Django, since the key relies on the
hash cached request's full URL
I think it would be nice for Django to provide helpers to invalidate:
* a specific url with specific query params, no matter the headers
* a specific url with any query params, for one or more specific headers
(among others)
* a group of urls
and the same list with a path instead of a full URL, in case the app uses
a single scheme, host, and port.
Problem is, we would need to have several distinct cache keys to be able
to do all this, which implies more keys, more calls to the cache and / or
bigger keys. Since some people may choose performance over cache control,
a good solution might be to make `_generate_cache_key` and `get_cache_key`
customizable.
Here is the code I wrote to implement some of these features, after
looking at `django.utils.cache` (I use a Redis cache with `django-redis`):
{{{#!python
import re
from typing import Dict, Optional
from django.conf import settings
from django.core.cache import cache
from django.core.handlers.wsgi import WSGIRequest
from django.utils.cache import get_cache_key
def invalidate_view_cache(
path: str,
*,
vary_headers: Optional[Dict[str, str]] = None,
key_prefix: Optional[str] = None,
invalidate_whole_prefix: bool = False,
) -> int:
"""
This function first creates a fake WSGIRequest to compute the cache
key.
The key looks like:
views.decorators.cache.cache_page.key_prefix.GET.0fcb3cd9d5b34c8fe83f615913d8509b.c4ca4238a0b923820dcc509a6f75849b
.en-us.UTC
The first hash corresponds to the full url (including query params),
the second to the header values
if invalid_whole_prefix is True, we will invalidate all the keys
matching the prefix
that also match the header hash and prefix.
This mean we will deletes all the keys returned by
views.decorators.cache.cache_page.key_prefix.GET.????????????????????????????????.c4ca4238a0b923820dcc509a6f75849b
.en-us.UTC
To be safe, we won't do it if key_prefix is None or ''
vary_headers should be a dict of every header used for this particular
view
In local environment, we have two defined renderers (default of DRF),
thus DRF adds
'Accept` to the Vary headers
Note: If LocaleMiddleware is used,
we'll need to use the same language code as the one in the cached
request
"""
request = WSGIRequest(
{
"PATH_INFO": path,
"REQUEST_METHOD": "GET",
"HTTP_HOST": settings.HOST,
"wsgi.input": None,
"wsgi.url_scheme":
f"http{'s'*settings.IS_DEPLOYED_ENVIRONMENT}",
}
)
if vary_headers:
request.META.update(vary_headers)
cache_key = get_cache_key(request, key_prefix=key_prefix)
if cache_key is None:
return 0
if invalidate_whole_prefix and key_prefix:
# specific to the redis implementation
keys = cache.keys(re.sub("[0-9a-f]{32}", "[0-9a-f]" * 32,
cache_key, 1))
return cache.delete_many(keys)
return cache.delete(cache_key)
}}}
I can try to work on a more generic implementation, if you think this idea
is worth it.
--
Ticket URL: <https://code.djangoproject.com/ticket/31938>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: Laurent Tramoy | Owner: nobody
Component: Core (Cache system) | Version: 3.1
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* status: new => closed
* resolution: => duplicate
* component: Utilities => Core (Cache system)
* stage: Unreviewed => Accepted
Comment:
Hi Laurent.
Thanks for the report. I was going to Accept this as a new feature — I've
spent a good amount of time rifling through cache keys to work out which
one to delete by hand. I think this would be a good addition.
**But** it looks to me like #5815 is the same issue, so let's close as a
duplicate.
Can I ask you take your idea to the DevelopersMailingList and ask for some
input of your thoughts for an implementation? I would be good if you want
to take this on to push it forwards! 👍
--
Ticket URL: <https://code.djangoproject.com/ticket/31938#comment:1>