[Django] #32531: Cache key not learnt for 304 responses — view must keep redoing work until "fresh" request comes in!
Django
"fresh" request comes in!
----------------------------------------------+------------------------
Reporter: Nathan Vander Wilt | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
----------------------------------------------+------------------------
Discovered an interesting situation that's easy to hit in a development
environment but might result in some wasted work in a production setting
too. Consider this situation:
* I load a Django-rendered page in my browser. This browser keeps its own
cache of the rendered response and its ETag.
* The Django cache on the serverside gets flushed somehow (e.g. if it's a
`LocMemCache` and I'm using `runserver` this happens whenever I touch a
file…)
* Now I (gently) referesh the page in my browser and it sends a request
with `If-None-Match` and the ETag that it knows for the page.
* Since the cache was flushed the view has to render the page again. In
this scenario the page hasn't changed, so after the render happens, the
ETag matches — hooray! The browser gets a 304 Not Modified response back.
But here's the problem: the next time I gently refresh the browser (i.e.
allow it to use its usual caching) the last step is repeated in whole!
That is, the view can end up rendering the page AGAIN and AGAIN and each
time the ETag matches in the end but Django does not seem to remember
that.
I suspect that the culprit might be related to the logic in
`UpdateCacheMiddleware.process_response` which will only call
`learn_cache_key` when the `response.status_code == 200`?
--
Ticket URL: <https://code.djangoproject.com/ticket/32531>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
"fresh" request comes in!
Reporter: Nathan Vander Wilt | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Nathan Vander Wilt):
Ah, I should note that my MIDDLEWARE ordering is:
{{{
[
'UpdateCacheMiddleware',
# … snip …
'ConditionalGetMiddleware',
# … snip …
'FetchFromCacheMiddleware'
]
}}}
Perhaps that is incorrect? Is ConditionalGet intended to come **after**
FetchFromCache or will that cause different problems?
--
Ticket URL: <https://code.djangoproject.com/ticket/32531#comment:1>
Django
"fresh" request comes in!
Reporter: Nathan Vander Wilt | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by Nathan Vander Wilt:
Old description:
> Discovered an interesting situation that's easy to hit in a development
> environment but might result in some wasted work in a production setting
> too. Consider this situation:
>
> * I load a Django-rendered page in my browser. This browser keeps its own
> cache of the rendered response and its ETag.
> * The Django cache on the serverside gets flushed somehow (e.g. if it's a
> `LocMemCache` and I'm using `runserver` this happens whenever I touch a
> file…)
> * Now I (gently) referesh the page in my browser and it sends a request
> with `If-None-Match` and the ETag that it knows for the page.
> * Since the cache was flushed the view has to render the page again. In
> this scenario the page hasn't changed, so after the render happens, the
> ETag matches — hooray! The browser gets a 304 Not Modified response back.
>
> But here's the problem: the next time I gently refresh the browser (i.e.
> allow it to use its usual caching) the last step is repeated in whole!
> That is, the view can end up rendering the page AGAIN and AGAIN and each
> time the ETag matches in the end but Django does not seem to remember
> that.
>
> I suspect that the culprit might be related to the logic in
> `UpdateCacheMiddleware.process_response` which will only call
> `learn_cache_key` when the `response.status_code == 200`?
New description:
Discovered an interesting situation that's easy to hit in a development
environment but might result in some wasted work in a production setting
too. Consider this situation:
* I load a Django-rendered page in my browser. This browser keeps its own
cache of the rendered response and its ETag.
* The Django cache on the serverside gets flushed somehow (e.g. if it's a
`LocMemCache` and I'm using `runserver` this happens whenever I touch a
file…)
* Now I (gently) referesh the page in my browser and it sends a request
with `If-None-Match` and the ETag that it knows for the page.
* Since the cache was flushed the view has to render the page again. In
this scenario the page hasn't changed, so after the render happens, the
ETag matches — hooray! The browser gets a 304 Not Modified response back.
But here's the problem: the next time I gently refresh the browser (i.e.
allow it to use its usual caching) the last step is repeated in whole!
That is, the view can end up rendering the page AGAIN and AGAIN and each
time the ETag matches in the end but Django does not seem to remember
that.
As soon as a "fresh" request comes in, i.e. a more aggressive refresh from
the browser that disables it's cache and yields a 200 response back, all
subsequent "gentle refresh" responses then get a 304 *without* the view
having to re-render.
I suspect that the culprit might be related to the logic in
`UpdateCacheMiddleware.process_response` which will only call
`learn_cache_key` when the `response.status_code == 200`?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/32531#comment:2>
Django
"fresh" request comes in!
Reporter: Nathan Vander Wilt | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Nathan Vander Wilt):
Ah, this might be invalid. While the `CacheMiddleware` does handle 200 vs.
304 somewhat differently throughout its own logic, my impression is that
the developer simply don't intend `ConditionalGetMiddleware` to work well
and it is the real culprit here.
The <https://docs.djangoproject.com/en/3.1/topics/conditional-view-
processing/#comparison-with-middleware-conditional-processing> discussion
referencing it warns:
> It doesn’t save you from generating the response, which may be
expensive.
and implies the goal of ± ETag handling is only much more modest such
that:
> […] the amount of network traffic sent back to the clients will still be
reduced if the view hasn’t changed.
Feel free to close as invalid if this is indeed the case.
--
Ticket URL: <https://code.djangoproject.com/ticket/32531#comment:3>
Django
"fresh" request comes in!
Reporter: Nathan Vander Wilt | Owner: nobody
Component: HTTP handling | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* status: new => closed
* resolution: => invalid
--
Ticket URL: <https://code.djangoproject.com/ticket/32531#comment:4>