[Django] #36957: Django psycopg connection pool + fork()

[Django]

#36957: Django psycopg connection pool + fork()
---------------------+-----------------------------------------
Reporter: doc | Type: Bug
Status: new | Component: Uncategorized
Version: 6.0 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------+-----------------------------------------
Django's PostgreSQL backend stores psycopg_pool.ConnectionPool objects in
a class-level dict (DatabaseWrapper._connection_pools). When gunicorn (or
any pre-forking server) forks worker processes, all children inherit
references to the same pool objects — and crucially, the same underlying
TCP sockets to PostgreSQL. Multiple workers then read/write the same
socket concurrently, corrupting the PostgreSQL wire protocol.


**Root cause**

{{{
# django/db/backends/postgresql/base.py
class DatabaseWrapper(BaseDatabaseWrapper):
_connection_pools = {} # class-level dict — survives fork()

@property
def pool(self):
if self.alias not in self._connection_pools:
pool = ConnectionPool(...)
self._connection_pools.setdefault(self.alias, pool)
return self._connection_pools[self.alias]
}}}

**Workaround**

{{{
# gunicorn.conf.py
def post_fork(server, worker):
from django.db.backends.postgresql.base import DatabaseWrapper
DatabaseWrapper._connection_pools.clear()
}}}

**Suggested fix**
Use `os.register_at_fork(after_in_child=...)` to clear `_connection_pools`
in child processes, or check `os.getpid()` in the `pool` property and
recreate when it differs from the creating process.

**Tested with**
Django 6.0.2
psycopg 3.2.x – 3.3.2
psycopg-pool 3.2.x – 3.3.0
gunicorn 25.x (--worker-class asgi)
Python 3.12 – 3.14
--
Ticket URL: <https://code.djangoproject.com/ticket/36957>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36957: Django psycopg connection pool + fork()

-------------------------------+--------------------------------------
Reporter: doc | Owner: (none)

Type: Bug | Status: new
Component: Uncategorized | Version: 6.0

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------+--------------------------------------
Changes (by doc):

* Attachment "django-pool-fork-bug.zip" added.

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------+--------------------------------------
Reporter: doc | Owner: (none)
Type: Bug | Status: new
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by doc):

* Attachment "django-pool-fork-bug.2.zip" added.

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------+--------------------------------------
Reporter: doc | Owner: (none)
Type: Bug | Status: new
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by doc):

* Attachment "django-pool-fork-bug.2.zip" removed.

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------+--------------------------------------
Reporter: doc | Owner: (none)
Type: Bug | Status: new
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------

Description changed by doc:

Old description:

New description:

Django's PostgreSQL backend stores `psycopg_pool.ConnectionPool` objects

in a class-level dict (`DatabaseWrapper._connection_pools`). When gunicorn

The minimal reproducible example project is in the attachments

--
--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:1>

[Django]

#36957: Django psycopg connection pool + fork()

-------------------------------------+-------------------------------------
Reporter: Anna | Owner: Dhruvan
| Gnanadhandayuthapani
Type: Bug | Status: assigned

Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Dhruvan Gnanadhandayuthapani):

* owner: (none) => Dhruvan Gnanadhandayuthapani
* stage: Unreviewed => Accepted
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:2>

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------------+-------------------------------------
Reporter: Anna | Owner: Dhruvan
| Gnanadhandayuthapani
Type: Bug | Status: assigned
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Dhruvan Gnanadhandayuthapani):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:3>

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------------+-------------------------------------
Reporter: Anna | Owner: Dhruvan
| Gnanadhandayuthapani
Type: Bug | Status: assigned
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Dhruvan Gnanadhandayuthapani):

Opened a PR that implements the `os.register_at_fork()` check along with a
flag. This solution was preferred over running `os.getpid()` in the pool
property, because it is wasteful to do it each time the property is
accessed.

https://github.com/django/django/pull/20790
--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:4>

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------------+-------------------------------------
Reporter: Anna | Owner: Dhruvan
| Gnanadhandayuthapani
Type: Bug | Status: assigned
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:

| Unreviewed

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by JaeHyuckSa):

* stage: Accepted => Unreviewed

Comment:

Hello Djruvan! The “Accept” status on a Django ticket should be set by a
triager after reviewing the ticket. If you haven’t had a chance to read
the contribution guidelines, I would appreciate it if you could take a
look.

- https://docs.djangoproject.com/en/dev/internals/contributing/triaging-
tickets
-
https://docs.djangoproject.com/en/dev/internals/contributing/?utm_source=chatgpt.com
--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:5>

[Django]

#36957: Django psycopg connection pool + fork()
-------------------------------------+-------------------------------------
Reporter: Anna | Owner: Dhruvan
| Gnanadhandayuthapani
Type: Bug | Status: assigned
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

I would say this is close if not a duplicate of #31637. The problem of
forking after connection creation is not specific to psycopg connection
pools.
--
Ticket URL: <https://code.djangoproject.com/ticket/36957#comment:6>