Re: [Django] #36964: Clarify how persistent connections interact with runserver

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Youssef Tarek Ali):

* has_patch: 0 => 1

Comment:

Replying to [comment:10 Tim Graham]:
> The quoted statement in the documentation was written by one of Django's
most esteemed contributors (2ee21d9f0d9eaed0494f3b9cd4b5bc9beffffae5).
While something may have changed in the intervening years, we need a more
rigorous explanation.
>
> I believe that multiple threads may be used by the built-in runserver,
even if requests are not concurrent. I verified this making this
modification:
> {{{
> diff --git a/django/db/backends/base/base.py
b/django/db/backends/base/base.py
> index 23015a57a3..94e7197dde 100644
> --- a/django/db/backends/base/base.py
> +++ b/django/db/backends/base/base.py
> @@ -51,6 +51,9 @@ class BaseDatabaseWrapper:
> queries_limit = 9000
>
> def __init__(self, settings_dict, alias=DEFAULT_DB_ALIAS):
> + print("DB Wrapper")
> + import threading
> + print(threading.get_ident())
> # Connection related attributes.
> # The underlying database connection.
> self.connection = None
> }}}
> And making several requests:
> {{{
> Starting WSGI development server at http://127.0.0.1:8000/
> Quit the server with CONTROL-C.
>
> DB Wrapper
> 139299259958976
> [14/Mar/2026 00:41:02] "GET /admin/ HTTP/1.1" 200 11255
> [14/Mar/2026 00:41:04] "GET /admin/polls/choice/ HTTP/1.1" 200 11343
> [14/Mar/2026 00:41:04] "GET /admin/jsi18n/ HTTP/1.1" 200 3342
> [14/Mar/2026 00:41:08] "GET /admin/polls/question/ HTTP/1.1" 200 13477
> DB Wrapper
> 139299250517696
> [14/Mar/2026 00:41:09] "GET /admin/jsi18n/ HTTP/1.1" 200 3342
> [14/Mar/2026 00:41:12] "GET /admin/polls/choice/ HTTP/1.1" 200 1134
> }}}
> I don't believe the AI-generated patch correctly explained the nuances
here.

Thanks for the logs and feedback, Natalia. I see now that my previous
rationale was off.

I wrote this update myself to make sure the technical details are correct.
The new PR explains that database connections are thread-local and that
there's no guarantee of thread reuse for sequential requests, even with
Keep-Alive. I also added a note about the auto-reloader.

The documentation builds correctly locally. Let me know if I've still
missed anything.

New PR: https://github.com/django/django/pull/20905
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

How is the development server's behavior different from multithreaded WSGI
servers used in production? Do they "guarantee that sequential requests
will use the same thread"?
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:12>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Sołtysik):

> The quoted statement in the documentation was written by one of Django's
most esteemed contributors (2ee21d9f0d9eaed0494f3b9cd4b5bc9beffffae5).
While something may have changed in the intervening years, we need a more
rigorous explanation.

The statement was probably correct when it was written, and what changed
later was the keep-alive support added around
https://github.com/django/django/pull/10609.

> How is the development server's behavior different from multithreaded
WSGI servers used in production? Do they "guarantee that sequential
requests will use the same thread"?

It seems that, unlike production servers, runserver generally keeps
creating new threads for each client, as mentioned earlier and confirmed
in a [https://forum.djangoproject.com/t/development-
server/24292/2#:~:text=The%20runserver%20command%20relies%20on%20socketserver.ThreadingMixIn%20which%20defaults%20to%20creating%20one%20thread%20per%20request
forum post]. But the threads are reused with HTTP keep-alive, which allows
persistent DB connections to work.
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:13>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by David Smith):

* has_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:14>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sushmita Yadav):

Hi, I would like to work on this issue. Could you please confirm if it's
available and guide me if there are any specific expectations?
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:15>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

Thanks. If helpful, I'll note the way contributions work around here, we
depend on you to study the issue and help us develop the acceptance
requirements together. There are very few tickets where we know exactly
what we want but stay quiet about it, leaving the details "as an exercise
for the contributor". Those are marked mostly with the ''Easy Pickings''
flag.
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:16>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by charankakumanu):

* has_patch: 0 => 1

Comment:

Hi, I’ve submitted a PR for this issue:
https://github.com/django/django/pull/21018

Feedback is welcome.
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:17>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by blighj):

I went to review the recent PR for this ticket and I don't think this
needs any change at all.

I rooted through the code and from my reading, runserver never reuses a
database connection across client connections.
`socketserver.ThreadingMixIn` starts one thread per accepted socket, and
database connections are thread-local, so a connection lives and dies with
the socket that created it.

The only bit of the docs that is slightly out of date is the "for each
request". That was true when written, there was no support for KeepAlive,
every request was a connection.

So yes now, a keep-alive connection will sometimes land on the same thread
and it'll look like DB connections are being shared. Which requests share
a socket is effectively nondeterministic from the developer's side, it
depends on how your browser happens to pool its connections, so you can't
rely on it. Open a second tab, return a StreamingHttpResponse (and I'm
sure there are more) and you'll get a different client connection, a
different thread and no sharing, plus an orphaned DB connection left open
until it's GC'd.

The docs are right to advise against using persistent connections with
runserver. I don't think they need updating, the only thing that I'd make
any case for is to change `creates a new thread for each request` to
`creates a new thread for each client connection`, but in a section about
database connections that overloads "connection" enough that I don't think
even that is worth it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:18>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Sołtysik):

Personally, I find persistent connections useful with `runserver`, as they
make navigating through my app smoother, so the sentence "Don’t enable
them during development" looks misleading. I think that a softer
"Depending on how your browser handles keep-alive, they may or may not
work for you" would do much better.
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:19>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by blighj):

Thanks for the feedback, I take your point. To be honest I've used it
myself in my last gig, for the same reasons, we had a shared db server
over the cloud used by our cms team (5 or 6 people max) and every so often
we'd all get a "sorry, too many clients already" error and all work was
halted until someone kicked the db server. The first few times it happened
it was painful as it took us a while to realise the db needed restarting.
We never figured out what was the cause and it was a rare occurrence,
dozen times a year, though often in clusters. We could see that the db
server was full of idle connections but couldn't figure out where they
were coming from. I now highly suspect it was the persistent connections.
But yeah we had it on more because we had it on in production and thought
it would help speed up dev without ever knowing about the docs
recommendation for runserver.
The question to decide is whether Django's reference documentation on
databases should give a soft recommendation to use something that was not
designed to work, may or may not work, and may cause a hard-to-diagnose
failure.
A community blog post, with lots of benchmarks and going into details on
the inner workings and tradeoffs, I can see that no problem. Something in
the Django docs, not so sure...

What if we said.
{{{#!diff
--- a/docs/ref/databases.txt
+++ b/docs/ref/databases.txt
@@ -89,9 +89,11 @@
to be reused. This will help keep the number of simultaneous connections
to
this database small.

-The development server creates a new thread for each request it handles,
-negating the effect of persistent connections. Don't enable them during
-development.
+The development server isn't designed for persistent connections. It
creates
+a new thread for each HTTP connection it handles, so database connections
+aren't reused between them and can be left orphaned. You may see one
reused
+while a client keeps an HTTP connection alive, but that's outside your
+control. Persistent connections shouldn't be enabled during development.

When Django establishes a connection to the database, it sets up
appropriate
parameters, depending on the backend being used. If you enable persistent
}}}

These seem like accurate claims for the docs to me. It acknowledges how
persistent connections may appear to work, points out that it isn't
designed, and gives a clue to the failure mode that would have helped my
old team. It also downgrades the `don't` to a `shouldn't`, if people want
to take the risk, that's on them. And it doesn't open up Django
maintainers to bug reports asking why it doesn't work reliably, or where
all these idle connections have come from.

To double check the claims were accurate I ran a little test against
Postgres, with 40 requests with and without keep alive:
||= =||= idle backends =||
|| baseline || 0 ||
|| 40 requests, 1 keep-alive connection || 1 ||
|| 40 requests, 40 separate connections || 41 ||
|| after `gc.collect()` || 0 ||

So you can see from the first pass there is a single connection serving
all 40 requests. The second creates one per client connection and leaves
them sitting idle in the database until they're garbage collected, (41
because the keep-alive connection from the first pass is still there too).
--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:20>

[Django]

#36964: Clarify how persistent connections interact with runserver
-------------------------------------+-------------------------------------
Reporter: Adam Sołtysik | Owner: Youssef
Type: | Tarek Ali
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by blighj):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36964#comment:21>