[Django] #36655: GzipMiddleware buffers streaming responses

[Django]

#36655: GzipMiddleware buffers streaming responses
-----------------------------------------+------------------------------
Reporter: Adam Johnson | Owner: Adam Johnson
Type: Bug | Status: assigned
Component: HTTP handling | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------------
Currently, `GzipMiddleware`, via `compress_sequence()`, buffers the entire
response before sending it to the client. This can cause issues for
clients that expect to receive data in chunks, such as those using Server-
Sent Events (SSE) or WebSockets.

This issue was reported to me in the django-browser-reload project back in
[https://github.com/adamchainz/django-browser-reload/pull/161 Issue #161
(2023)], where a contributor fixed it with a workaround, and I didn't
think to investigate. Now, while implementing
[https://github.com/adamchainz/django-http-compression django-http-
compression], I have realized that it’s a proper bug that can be fixed by
adding a call to `zfile.flush()`, as done in its
[https://github.com/adamchainz/django-http-compression/pull/8 PR #8].

To reproduce the issue, use the below app, which can be run with `uv run
--script`. If you comment out `GzipMiddleware` and load the page in a
browser, you will see the numbers incrementing every second. If you
include `GzipMiddleware`, the page will never load. Adding the
`zfile.flush()` call in `compress_sequence()` fixes the issue.

{{{#!python
#!/usr/bin/env uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = [
# "django",
# ]
# ///
from __future__ import annotations

import os
import sys
import time

from django.conf import settings
from django.core.wsgi import get_wsgi_application
from django.http import StreamingHttpResponse
from django.urls import path

settings.configure(
# Dangerous: disable host header validation
ALLOWED_HOSTS=["*"],
# Use DEBUG=1 to enable debug mode
DEBUG=(os.environ.get("DEBUG", "") == "1"),
# Make this module the urlconf
ROOT_URLCONF=__name__,
# Only gzip middleware
MIDDLEWARE=[
"django.middleware.gzip.GZipMiddleware",
],
)


def clock(request):
def stream():
yield "<h1>Clock</h1>\n"
count = 1
while True:
yield f"<p>{count}</p>\n"
count += 1
time.sleep(1)

return StreamingHttpResponse(stream())


urlpatterns = [
path("", clock),
]

app = get_wsgi_application()

if __name__ == "__main__":
from django.core.management import execute_from_command_line

execute_from_command_line(sys.argv)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36655>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.