[Django] #35051: HEAD Responses Drop Headers
Django
-----------------------------------------+------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 5.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
When using runserver headers are dropped for head requests, in particular
content-length. Because my HEAD request is serving a large body content, I
do not include it in the body since it is just dropped.
Headers from runserver:
{{{#!python
Headers({'date': 'Tue, 19 Dec 2023 12:52:23 GMT', 'server':
'WSGIServer/0.2 CPython/3.11.2', 'accept-ranges': 'bytes', 'content-type':
'text/html; charset=utf-8', 'x-frame-options': 'DENY', 'x-content-type-
options': 'nosniff', 'referrer-policy': 'same-origin', 'cross-or
igin-opener-policy': 'same-origin'})
}}}
Headers from uvicorn asgi server:
{{{#!python
Headers({'date': 'Tue, 19 Dec 2023 12:54:49 GMT', 'server': 'uvicorn',
'accept-ranges': 'bytes', 'content-length': '121283919', 'content-type':
'text/html; charset=utf-8', 'x-frame-options': 'DENY', 'x-content-type-
options': 'nosniff', 'referrer-policy': 'same-origin', 'cross-origin-
opener-policy': 'same-origin'})
}}}
Notice the uvicorn properly includes the content-length header that was
set in the view.
View source snippet:
{{{#!python
if request.method == 'HEAD':
print('HEAD Request')
return HttpResponse(headers={
'Accept-Ranges': 'bytes',
'Content-Length': str(file_size)
})
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35051>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 5.0
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Natalia Bidart):
* cc: Sarah Boyce, Jannik Schürg (added)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
I did some research on the topic and from [https://www.rfc-
editor.org/rfc/rfc9110.html#name-content-length the corresponding RFC] it
seems that this report is valid and should be accepted:
> A server MAY send a Content-Length header field in a response to a HEAD
request (Section 9.3.2); a server MUST NOT send Content-Length in such a
response unless its field value equals the decimal number of octets that
would have been sent in the content of a response if the same request had
used the GET method.
The removal of the `Content-Length` seems to be located in this code:
{{{#!diff
diff --git a/django/core/servers/basehttp.py
b/django/core/servers/basehttp.py
index 6afe17cec4..e327974708 100644
--- a/django/core/servers/basehttp.py
+++ b/django/core/servers/basehttp.py
@@ -131,11 +131,6 @@ class ServerHandler(simple_server.ServerHandler):
def cleanup_headers(self):
super().cleanup_headers()
- if (
- self.environ["REQUEST_METHOD"] == "HEAD"
- and "Content-Length" in self.headers
- ):
- del self.headers["Content-Length"]
# HTTP/1.1 requires support for persistent connections. Send
'close' if
# the content length is unknown to prevent clients from reusing
the
# connection.
}}}
This code was added while fixing ticket #28054, and while it's correct not
to return the body of the response, it seems that the `Content-Length`
should be kept.
Regression in 8acc433e415cd771f69dfe84e57878a83641e78b
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:1>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Mariusz Felisiak):
I don't agree, we intentionally drop the `Content-Length`. Please check
the entire discussion in PR, e.g. this
[https://github.com/django/django/pull/16502#issuecomment-1405617633
comment]. As far as I'm aware, the current implementation is RFC
compliant. I'd mark this ticket as invalid.
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:2>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
* cc: Nick Pope (added)
* component: Uncategorized => HTTP handling
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:3>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Paul Bailey):
The use case for this is that your GET request has a large body and so you
do not want to have to produce two large bodies, one for the HEAD, one for
GET. Instead you want to match the exact headers a GET request would have
without producing the body content for the HEAD request. This is essential
for producing HTTP Range Requests in Django without a lot of overhead.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
As per the docs above, a HTTP HEAD should return Content-Length
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:4>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Paul Bailey):
The discussion linked is over Content-Length being returned for the body
of the HEAD response, so this is always "Content-Length: 0" which doesn't
match the GET request.
The easiest thing to do at the time was to just remove Content-Length
since it is not required. However, for HTTP Range Requests Content-Length
is required.
So I think the proper thing to do is to allow it if Content-Length is not
Zero. If the header is not 0 then that means it was set by the user and
should be allowed through.
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:5>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Paul Bailey):
something like:
{{{#!python
def cleanup_headers(self):
super().cleanup_headers()
if (
self.environ["REQUEST_METHOD"] == "HEAD"
and "Content-Length" in self.headers
and str(self.headers["Content-Length"]) == "0"
):
del self.headers["Content-Length"]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:6>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Mariusz Felisiak):
As for me this is a new feature request for supporting HTTP ranges.
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:7>
Django
---------------------------------+------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: Bug | Status: new
Component: HTTP handling | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Comment (by Paul Bailey):
Replying to [comment:7 Mariusz Felisiak]:
> As for me this is a new feature request for supporting HTTP ranges.
I would imagine Content-Length in HEAD requests is also needed for other
streaming mechanisms.
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:8>
Django
Reporter: Paul Bailey | Owner: nobody
Component: HTTP handling | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Natalia Bidart):
* status: new => closed
* severity: Release blocker => Normal
* resolution: => invalid
* version: 5.0 => dev
* type: Bug => New feature
* stage: Accepted => Unreviewed
Comment:
Replying to [comment:2 Mariusz Felisiak]:
> I don't agree, we intentionally drop the `Content-Length`. Please check
the entire discussion in PR, e.g. this
[https://github.com/django/django/pull/16502#issuecomment-1405617633
comment]. As far as I'm aware, the current implementation is RFC
compliant. I'd mark this ticket as invalid.
Thank you Mariusz for the pointer to the specific message from Nick, it
provides a very complete and clear reasoning for the change.
With that in mind, I agree that this is not a valid **bug**. Additionally,
I agree that the optional return of `Content-Length` for `HEAD` requests,
enabling specific use cases, should be approached as a new feature, which
should be presented and discussed in the
[https://forum.djangoproject.com/c/internals/5 Django Forum] (following
[https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-
features/#requesting-features the documented guidelines for requesting
features]). Paul, would you be willing to start a new topic explaining the
current situation and outlining potential use cases for the feature?
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:9>
Django
-------------------------------+--------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: New feature | Status: closed
Component: HTTP handling | Version: dev
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Comment (by Paul Bailey):
sounds good
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:10>
Django
-------------------------------+--------------------------------------
Reporter: Paul Bailey | Owner: nobody
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
Reopening based on discussion at: https://forum.djangoproject.com/t
/optionally-do-not-drop-content-length-for-head-requests/26305
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:11>
Django
-------------------------------+--------------------------------------
Reporter: Paul Bailey | Owner: nobody
Type: New feature | Status: new
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by Paul Bailey):
* has_patch: 0 => 1
Comment:
PR ready for review:
https://github.com/django/django/pull/17643#issuecomment-1869135570
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:12>
Django
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Paul Bailey):
* owner: nobody => Paul Bailey
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:13>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
Changes (by Paul Bailey):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:14>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
* stage: Accepted => Unreviewed
Comment:
You cannot accept your own tickets.
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:15>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
* cc: HAMA Barhamou (added)
* stage: Unreviewed => Accepted
Comment:
Hello, after reading all the comments on the tiket, I think we can accept
it and start revising the proposed path. Merry Christmas to all Christians
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:16>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
Changes (by HAMA Barhamou):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:17>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
* cc: Florian Apolloner (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:18>
Django
-------------------------------+---------------------------------------
Reporter: Paul Bailey | Owner: Paul Bailey
Type: New feature | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+---------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:19>
Django
Reporter: Paul Bailey | Owner: Paul
Cleanup/optimization | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* type: New feature => Cleanup/optimization
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:20>
Django
-------------------------------------+-------------------------------------
Reporter: Paul Bailey | Owner: Paul
Type: | Bailey
Cleanup/optimization | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:21>
Django
-------------------------------------+-------------------------------------
Reporter: Paul Bailey | Owner: Paul
Type: | Bailey
Cleanup/optimization | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:22>
Django
-------------------------------------+-------------------------------------
Reporter: Paul Bailey | Owner: Paul
Type: | Bailey
Cleanup/optimization | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:23>
Django
-------------------------------------+-------------------------------------
Reporter: Paul Bailey | Owner: Paul
Type: | Bailey
Component: HTTP handling | Version: dev
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"9d52e0720f79ac3cff3d9888a97ac227884a621e" 9d52e072]:
{{{
#!CommitTicketReference repository=""
revision="9d52e0720f79ac3cff3d9888a97ac227884a621e"
Fixed #35051 -- Prevented runserver from removing non-zero Content-Length
for HEAD requests.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35051#comment:24>