[Django] #36656: `GZipMiddleware` drops content from async streaming responses

[Django]

#36656: `GZipMiddleware` drops content from async streaming responses
-----------------------------------------+------------------------------
Reporter: Adam Johnson | Owner: Adam Johnson
Type: Bug | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------------
0bd2c0c9015b53c41394a1c0989afbfd94dc2830 (#33735) expanded
`GzipMiddleware` to support async streaming responses. But it does so with
a faulty `gzip_wrapper()` that compresses chunks as individual files,
rather than as a continuous stream. As a result, only the first chunk is
decompressible, and browsers drop the rest of the response, even as they
stay connected and download all the data.

The solution is to use a streaming approach with `GzipFile`, as is already
done for sync responses in `compress_sequence()`.

Additionally, the sync approach currently starts by sending an empty chunk
to flush the headers. I think that may be necessary for async responses
too, since the first content chunk may take an arbitrary amount of time to
be generated.

To reproduce the issue, use the app below, which can be run with `uv run
--script`. If you comment out `GzipMiddleware` and load the page in a
browser, you will see the numbers incrementing every second. If you
include `GzipMiddleware`, only the header will appear, and the rest of the
response will be dropped.

{{{#!python
#!/usr/bin/env uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = [
# "daphne",
# "django",
# ]
# ///
from __future__ import annotations

import asyncio
import os
import sys

from django.conf import settings
from django.core.asgi import get_asgi_application
from django.http import StreamingHttpResponse
from django.urls import path

settings.configure(
# Dangerous: disable host header validation
ALLOWED_HOSTS=["*"],
# Use DEBUG=1 to enable debug mode
DEBUG=(os.environ.get("DEBUG", "") == "1"),
# Make this module the urlconf
ROOT_URLCONF=__name__,
# Use Daphne for async runserver
INSTALLED_APPS=[
"daphne",
],
ASGI_APPLICATION=f"{__name__}.app",
# Only gzip middleware
MIDDLEWARE=[
"django.middleware.gzip.GZipMiddleware",
],
)


async def clock(request):
async def stream():
yield "<h1>Clock</h1>\n"
count = 1
while True:
yield f"<p>{count}</p>\n"
count += 1
await asyncio.sleep(1)

return StreamingHttpResponse(stream())


urlpatterns = [
path("", clock),
]

app = get_asgi_application()

if __name__ == "__main__":
from django.core.management import execute_from_command_line

execute_from_command_line(sys.argv)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36656>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.