[Django] #35059: "Too many connections" regression from 4.2->5.0 - asgi server leaves stale DB connections

[Django]

#35059: "Too many connections" regression from 4.2->5.0 - asgi server leaves stale
DB connections
-------------------------------------------+------------------------
Reporter: James Thorniley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 5.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------------+------------------------
This seems to be a regression from 4.2 to 5.0.

We are using Django ASGI server via uvicorn, with a mariadb database. When
updating our app servers from Django 4.2.8 to 5.0, we experienced a spike
in the number of database connections, eventually going over the limit set
for the DB and causing 500 errors due to

`django.db.utils.OperationalError: (1040, 'Too many connections')`

I've created a minimal project to reproduce what I think is the phenomenon
here using local docker container for the DB:

https://github.com/jthorniley/django-asgi

I've put complete reproduction steps in that github link, but I will
summarise here:

* When running a Django 5.0 ASGI server, some connections are not closed
when requests complete, leaving stale connections in the DB. This can be
confirmed by running curl against a local server and then `show
processlist` in the mysql CLI when the requests are complete.
* As a result, if you send a moderate number of parallel requests to the
server, it will go past the limit of DB connections (I think the out-of-
the-box limit for the mariadb docker container is around 150, but it is
configurable). Then new connections will fail with the `Too many
connections` error. Obviously its theoretically possible to go over this
limit anyway but it happens sooner due to server processes not cleaning up
connections after requests complete.
* You can downgrade to 4.2.8 and run the same reproduction project and see
that there is no issue there. The DB connections are closed after every
individual request and hence provided there aren't more parallel requests
than allowed connections to the DB you are safe.

--
Ticket URL: <https://code.djangoproject.com/ticket/35059>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35059: "Too many connections" regression from 4.2->5.0 - asgi server leaves stale
DB connections

---------------------------------+--------------------------------------

Reporter: James Thorniley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 5.0

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+--------------------------------------

Comment (by bjd183):

I also experienced this with Postgres. Downgraded to Django 4.2.8

--
Ticket URL: <https://code.djangoproject.com/ticket/35059#comment:1>

[Django]

#35059: "Too many connections" regression from 4.2->5.0 - asgi server leaves stale
DB connections
---------------------------------+------------------------------------

Reporter: James Thorniley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 5.0
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Changes (by AryanGHM):

* stage: Unreviewed => Accepted

Comment:

I also could reproduce this bug using James's repo.

--
Ticket URL: <https://code.djangoproject.com/ticket/35059#comment:2>