[Django] #37177: Performance issue in Async Middleware handling.

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+-----------------------------------------
Reporter: Carlton Gibson | Type: Bug
Status: new | Component: HTTP handling
Version: 6.0 | Severity: Normal
Keywords: async | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+-----------------------------------------
Ticket #31224 in commit fc0fa72ff4cdbf5861a366e31cb8bbacd44da22d "Added
support for asynchronous views and middleware."

As part of this it added the `sync_capable`/`async_capable` flags to
`MiddlewareMixin` and made default middleware advertise async capability
via `__acall__`:

{{{
class MiddlewareMixin:
sync_capable = True
async_capable = True

...

async def __acall__(self, request):
"""
Async version of __call__ that is swapped in when an async request
is running.
"""
response = None
if hasattr(self, 'process_request'):
response = await sync_to_async(self.process_request)(request)
response = response or await self.get_response(request)
if hasattr(self, 'process_response'):
response = await sync_to_async(self.process_response)(request,
response)
return response
}}}

This (unwittingly) undermined the process in `load_middleware` to minimise
the
switches between sync and async contexts.

Every middleware in the default `startproject` stack
(`SecurityMiddleware`,
`SessionMiddleware`, `CommonMiddleware`, `CsrfViewMiddleware`,
`AuthenticationMiddleware`, `MessageMiddleware`,
`XFrameOptionsMiddleware`) is
a `MiddlewareMixin` subclass that declares `async_capable = True` but
implements its `process_request`/`process_response`/ `process_view` hooks
as
plain synchronous methods. (`RemoteUserMiddleware` is the lone built-in
written
natively async, and it is not in the default stack.)

The middleware chain is built as all async but each middleware then
re-introduces a boundary crossing for each hook — it wraps every sync
`process_request`/`process_response` in its own
`sync_to_async(thread_sensitive=True)`.

The net effect inverts the optimiser's intent: a single O(1) bracketing of
the
contiguous sync block becomes O(N) per-hook hopping onto the per-request
thread
and back.

We end up with essentially 16 context switches before reaching the view.

If, instead, the default middleware are marked as `async_capable = False`,
we get only 1 such transition during the middleware, as was the intent of
the original feature.

Asides:

* We get a second transition for a sync view, as that's wrapped in
`sync_to_async`.
* We only get 0 transitions if the middleware chain is totally native
async.

Both of these are expected, and within the performance profile we'd
expect.
Django views pretty much always hit the DB, and at that point the
additional
thread is already in play. A single sync transition during middle
processing is
going to be fine for most use case.

A quick benchmark driving ASGIHangler with Python 3.13, asgiref 3.11, on
Django
main (6.2.dev), with default middleware marked as async_capable vs not
(and an
`async-io` view doing no more than an `await asyncio.sleep(0.005)`) shows
significant throughput differences under load:

{{{
c=50 seq us/req conc req/s peak thr
async mw / /sync/ 1188.9 1331 51
sync mw / /sync/ 291.6 5080 51
async mw / /async/ 1115.1 1508 51
sync mw / /async/ 218.9 6448 51
async mw / /async-io/ 8156.2 1381 51
sync mw / /async-io/ 6684.5 3984 51

c=200 seq us/req conc req/s peak thr
async mw / /sync/ 1109.2 898 201
sync mw / /sync/ 400.9 4647 201
async mw / /async/ 1047.8 1436 201
sync mw / /async/ 212.1 6551 201
async mw / /async-io/ 8946.1 1411 201
sync mw / /async-io/ 7111.8 5448 201
}}}

**Executive summary*: `MiddlewareMixin` should not declare the default
middleware as `async_capable`.

I'm not sure we can just flip the flag — that's what I did for the
benchmark — but maybe MiddlewareMixin could check to see if
process_request/process_response were coroutine function or not before
just declaring `True`? (There's probably a little more due diligence to do
there too.)

I want to thank Mykhailo Havelia for pointing this issue out. The
conclusion to remove the async support entirely goes too far I think, but
we absolutely shouldn't be transitioning contexts multiple times each way
in this case.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+--------------------------------------

Reporter: Carlton Gibson | Owner: (none)

Type: Bug | Status: new
Component: HTTP handling | Version: 6.0

Severity: Normal | Resolution:

Keywords: async | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+--------------------------------------

Description changed by Carlton Gibson:

Old description:

New description:

main (6.2.dev), with middleware marked as async_capable vs not (and an

--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:1>

[Django]

> main (6.2.dev), with middleware marked as async_capable vs not (and an

I want to thank Mykhailo Havelia for pointing this issue out. We

absolutely shouldn't be transitioning contexts multiple times each way in
this case.

--
--

Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:2>

[Django]

> I want to thank Mykhailo Havelia for pointing this issue out. We
> absolutely shouldn't be transitioning contexts multiple times each way in
> this case.

We end up with essentially 16 context switches reaching the view and back.

--
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:3>

[Django]

I want to thank Mykhailo Havelia for pointing this issue out. We
absolutely shouldn't be transitioning contexts multiple times each way in
this case.

--
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:4>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+--------------------------------------
Reporter: Carlton Gibson | Owner: (none)
Type: Bug | Status: new
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+--------------------------------------

Changes (by Mykhailo Havelia):

* cc: Mykhailo Havelia (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:5>

[Django]

#37177: Performance issue in Async Middleware handling.

--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned

Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:

Keywords: async | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------+---------------------------------------
Changes (by Jacob Walls):

* owner: (none) => Jacob Walls
* stage: Unreviewed => Accepted
* status: new => assigned

Comment:

This was mentioned on the [https://forum.djangoproject.com/t/sync-to-
async-called-14-times-with-default-django-middleware-and-2-times-with-no-
middleware/31701 forum] at least once, so thanks for the confirmation that
this runs counter to the original design.

> I'm not sure we can just flip the flag — that's what I did for the
benchmark — but maybe MiddlewareMixin could check to see if
process_request/process_response were coroutine function or not before

just declaring True? (There's probably a little more due diligence to do
there too.)

Just flipping the flag is what a user did, when they saw this in their own
middleware, as for them it was a regression, see
[https://forum.djangoproject.com/t/issues-with-middleware-in-django-3-1
-using-asgi/4080/3 forum], but given how many releases have followed,
introspecting seems like the most backward-compatible thing we can do
right now.

Separately, I have a question about how much longer we plan to maintain
`MiddlewareMixin`.

Tentatively assigning to myself, to see if we can advance this before next
Wednesday's bug freeze for 6.1.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:6>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Carlton Gibson):

@Jacob: great thanks! (I'm off on holiday so won't have time to process it
instantly)

My half-thought was, if introspection is too clever:

1. Manually mark (affected) built in middleware as `async_capable = False`
2. Deprecate `MiddlewareMixin.__acall__` implementation.

And that might be sufficient?

Tests: Locally I patched sync_to_async and async_to_sync calls to count
invocations, but that would need to be run alone. 🤔
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:7>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Mykhailo Havelia):

Replying to [comment:7 Carlton Gibson]:

> @Jacob: great thanks! (I'm off on holiday so won't have time to process
it instantly)
>
> My half-thought was, if introspection is too clever:
>
> 1. Manually mark (affected) built in middleware as `async_capable =
False`
> 2. Deprecate `MiddlewareMixin.__acall__` implementation.
>
> And that might be sufficient?
>

That's a viable option, but it could break things for people who inherit
from the default middleware and rely on async_capable. We'd need to
double-check that.

https://github.com/django/new-features/issues/102 is another option for
solving this for CPU-bound middleware.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:8>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Natalia Bidart):

As part of other async benchmarking work I'm doing, I took some time to
measure the problem from this ticket, and got consistent results. On a
real ASGI server end-to-end (daphne, single process), using the default 7
middlewares, and an async view returning immediately, Python 3.14.5 and
asgiref 3.11.1, I tested with `ab` across concurrency 10 to 200 (`-n
8000`):

{{{
-c │ async_capable=True │ async_capable=False
────┼─────────────────────┼─────────────────────
10 │ 576 req/s │ 920 req/s
50 │ 587 req/s │ 926 req/s
100 │ 570 req/s │ 933 req/s
200 │ 533 req/s │ 883 req/s
}}}

Throughput stays flat and latency grows linearly in both (I ran this in my
laptop, a saturated single process), so the change shows up as the
throughput **ceiling:** marking the built-in middleware sync-only lifts it
~1.6x at every load level (~1.7 ms to ~1.1 ms per request, WSGI reference,
same view: ~1690 req/s.)
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:9>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Mykhailo Havelia):

Replying to [comment:9 Natalia Bidart]:

> Throughput stays flat and latency grows linearly

In production I observed somewhat different behaviour. The convoy effect
was noticeably stronger under CPU-bound work. We found that past ~40% CPU
utilisation, response-time degradation became non-linear, so we had to
keep CPU usage below that threshold.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:10>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Natalia Bidart):

Replying to [comment:10 Mykhailo Havelia]:

> In production I observed somewhat different behaviour. The convoy effect
was noticeably stronger under CPU-bound work. We found that past ~40% CPU
utilisation, response-time degradation became non-linear, so we had to
keep CPU usage below that threshold.

That's a useful data point, thanks. I think non-linear degradation is
consistent with GIL contention getting worse as CPU work rises, which my
benchmark setup wouldn't capture: the view I used does no application
work, so it really measures the framework/middleware floor. If you can
share a benchmark in any reproducible form, I'd really like to measure the
CPU-bound case directly rather than infer it.

For reference, here's how I got mine. Single process each, default
middleware stack, an async view that returns immediately, loaded with
ApacheBench:
{{{
# WSGI for baseline sync, other port
gunicorn -b 127.0.0.1:9000 -w 1 mysite.wsgi:application

# ASGI
daphne -b 127.0.0.1 -p 9001 mysite.asgi:application

# concurrent requests (change -c 10/50/100/200)
ab -n 8000 -c 50 http://127.0.0.1:9001/<async-view>/
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:11>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Mykhailo Havelia):

I'll send everything I have to Discord
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:12>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Jacob Walls):

Replying to [comment:7 Carlton Gibson]:

> 1. Manually mark (affected) built in middleware as `async_capable =
False`
> 2. Deprecate `MiddlewareMixin.__acall__` implementation.
>
> And that might be sufficient?

I think we should just do 1., as IMO it's a bug in the core middleware,
and it's a reasonable user-duty to flip the flag when subclassing to
extend with async functionality. Small release note.

I don't see why we have to do 2., `MiddlewareMixin` is public and
`__acall__` seems useful.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:13>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Natalia Bidart):

Replying to [comment:12 Mykhailo Havelia]:

> I'll send everything I have to Discord

Can you please not use Discord and attach to this ticket instead? Or post
in a new forum thread. Discord is not formally used for discussing
development of Django, it's mostly used for informal chat and assisting
how to use Django.

The right channels for developing Django are this ticket Trac system and
the [https://forum.djangoproject.com/c/internals/5 Official Django Forum].
Thank you!
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:14>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------

Comment (by Mykhailo Havelia):

Replying to [comment:14 Natalia Bidart]:

> The right channels for developing Django are this ticket Trac system and
the [https://forum.djangoproject.com/c/internals/5 Official Django Forum].
Thank you!

I'm not ready to send it to the forum yet because it's still a bit messy
and not finished. So I can't guarantee a timeframe for when it'll be done.
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:15>

[Django]

#37177: Performance issue in Async Middleware handling.
--------------------------------+---------------------------------------
Reporter: Carlton Gibson | Owner: Jacob Walls
Type: Bug | Status: assigned
Component: HTTP handling | Version: 6.0
Severity: Normal | Resolution:
Keywords: async | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+---------------------------------------
Comment (by Mykhailo Havelia):

Replying to [comment:13 Jacob Walls]:

> Replying to [comment:7 Carlton Gibson]:
> > 1. Manually mark (affected) built in middleware as `async_capable =
False`
> > 2. Deprecate `MiddlewareMixin.__acall__` implementation.
> >
> > And that might be sufficient?
>
> I think we should just do 1., as IMO it's a bug in the core middleware,
and it's a reasonable user-duty to flip the flag when subclassing to
extend with async functionality. Small release note.
>
> I don't see why we have to do 2., `MiddlewareMixin` is public and
`__acall__` seems useful.

I was worried about two things.

- Backwards compatibility, for people who extend default Django
middlewares and rely on async_capable and/or __acall__.
- People who don't want to use threads but still want to use at least the
default CPU-bound Django middlewares.

To avoid breaking backwards compatibility, I couldn't find a better
solution than providing dedicated async middlewares. Anyway, we can split
it and do it step by step
--
Ticket URL: <https://code.djangoproject.com/ticket/37177#comment:16>