[Django] #36946: Running tests on SQLite with --parallel (using spawn) does not respect DATABASES["TEST"]["NAME"]
4 views
Skip to first unread message
Django
Feb 24, 2026, 10:39:50 AM (2 days ago) Feb 24
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Type: Bug
Status: new | Component: Testing
| framework
Version: dev | Severity: Normal
Keywords: tests, sqlite, | Triage Stage:
parallel | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
When running unit tests on SQLite on macOS with the `--parallel`
[https://docs.djangoproject.com/en/6.0/ref/django-admin/#cmdoption-test-
parallel flag] and the `DATABASES["TEST"]["NAME"]`
[https://docs.djangoproject.com/en/6.0/ref/settings/#std-setting-TEST_NAME
setting] set (e.g. to `mytests.db`), the tests fail to run because SQLite
tries to connect to databases named `default_*.sqlite3` instead of the
cloned `mytests_*.db`.
You can replicate this with the Django test suite itself, e.g. with the
following change:
{{{#!diff
diff --git a/tests/test_sqlite.py b/tests/test_sqlite.py
index 0e6b0f672a..77981218fd 100644
--- a/tests/test_sqlite.py
+++ b/tests/test_sqlite.py
@@ -29,3 +29,9 @@ PASSWORD_HASHERS = [
]
USE_TZ = False
+
+import os
+
+DATABASE_NAME = os.getenv("DATABASE_NAME")
+if DATABASE_NAME:
+ DATABASES["default"]["TEST"] = {"NAME": DATABASE_NAME}
}}}
and then run a subset of the tests, e.g.
{{{#!shell
DATABASE_NAME=django.db ./runtests.py --verbosity=2 --parallel=2 --
model_fields
}}}
You'll get a bunch of errors like the following:
{{{#!shell
Traceback (most recent call last):
File "/../python/3.14.2/lib/python3.14/multiprocessing/process.py", line
320, in _bootstrap
self.run()
~~~~~~~~^^
File "/../python/3.14.2/lib/python3.14/multiprocessing/process.py", line
108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/../python/3.14.2/lib/python3.14/multiprocessing/pool.py", line
109, in worker
initializer(*initargs)
~~~~~~~~~~~^^^^^^^^^^^
File "/django/django/test/runner.py", line 479, in _safe_init_worker
init_worker(counter, *args, **kwargs)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/django/django/test/runner.py", line 464, in _init_worker
connection.creation.setup_worker_connection(_worker_id)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/django/django/db/backends/sqlite3/creation.py", line 145, in
setup_worker_connection
source_db = self.connection.Database.connect(
f"file:{alias}_{_worker_id}.sqlite3?mode=ro", uri=True
)
sqlite3.OperationalError: unable to open database file
}}}
but you can see that Django successfully created/destroyed the test
databases with the correct names:
{{{#!shell
Cloning test database for alias 'default' ('django.db')...
Cloning test database for alias 'default' ('django.db')...
# test results go here
Destroying test database for alias 'default' ('django_1.db')...
Destroying test database for alias 'default' ('django_2.db')...
Destroying test database for alias 'default' ('django.db')...
}}}
This also affects a multi-database setup if you set the `["TEST"]["NAME"]`
for that database. With the above example, if you run the
`multiple_database` test suite (instead of `model_fields`), you won't see
any errors for the `other` database; as Django will create and connect to
the fallback test name based on the alias, i.e. `other_*.sqlite3`.
However, if you also add `DATABASES["other"]["TEST"] = {"NAME": "other_" +
DATABASE_NAME}` to the settings changes in the above example, the same
error now also happens for the `other` database.
This prevents the use of custom names for running tests with `--parallel`
> 1 (or `auto`), which can be particularly useful when you want to use
`--keepdb` with different names. With `--parallel=1`, this issue does not
occur.
I believe it is because despite the cloning was done correctly using the
given test database file name in
[https://github.com/django/django/blob/2de474dffca008038a7f9fe10289f390c5f4f15b/django/db/backends/sqlite3/creation.py#L58-L60
get_test_db_clone_settings], the logic in
[https://github.com/django/django/blob/2de474dffca008038a7f9fe10289f390c5f4f15b/django/db/backends/sqlite3/creation.py#L145-L147
setup_worker_connection] does not use the clone db names (it always uses
`{alias}_{_worker_id}.sqlite3` instead) when copying them over to the in-
memory database.
FWIW, I encountered this while trying to run
[https://github.com/wagtail/wagtail/pull/13922 Wagtail's test suite] in
parallel with `--keepdb`. I can give a smaller reproducible example if
necessary, but I think the example with Django's own test suite can be a
good example too.
--
Ticket URL: <https://code.djangoproject.com/ticket/36946>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Type: Bug
Status: new | Component: Testing
| framework
Version: dev | Severity: Normal
Keywords: tests, sqlite, | Triage Stage:
parallel | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
When running unit tests on SQLite on macOS with the `--parallel`
[https://docs.djangoproject.com/en/6.0/ref/django-admin/#cmdoption-test-
parallel flag] and the `DATABASES["TEST"]["NAME"]`
[https://docs.djangoproject.com/en/6.0/ref/settings/#std-setting-TEST_NAME
setting] set (e.g. to `mytests.db`), the tests fail to run because SQLite
tries to connect to databases named `default_*.sqlite3` instead of the
cloned `mytests_*.db`.
You can replicate this with the Django test suite itself, e.g. with the
following change:
{{{#!diff
diff --git a/tests/test_sqlite.py b/tests/test_sqlite.py
index 0e6b0f672a..77981218fd 100644
--- a/tests/test_sqlite.py
+++ b/tests/test_sqlite.py
@@ -29,3 +29,9 @@ PASSWORD_HASHERS = [
]
USE_TZ = False
+
+import os
+
+DATABASE_NAME = os.getenv("DATABASE_NAME")
+if DATABASE_NAME:
+ DATABASES["default"]["TEST"] = {"NAME": DATABASE_NAME}
}}}
and then run a subset of the tests, e.g.
{{{#!shell
DATABASE_NAME=django.db ./runtests.py --verbosity=2 --parallel=2 --
model_fields
}}}
You'll get a bunch of errors like the following:
{{{#!shell
Traceback (most recent call last):
File "/../python/3.14.2/lib/python3.14/multiprocessing/process.py", line
320, in _bootstrap
self.run()
~~~~~~~~^^
File "/../python/3.14.2/lib/python3.14/multiprocessing/process.py", line
108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/../python/3.14.2/lib/python3.14/multiprocessing/pool.py", line
109, in worker
initializer(*initargs)
~~~~~~~~~~~^^^^^^^^^^^
File "/django/django/test/runner.py", line 479, in _safe_init_worker
init_worker(counter, *args, **kwargs)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/django/django/test/runner.py", line 464, in _init_worker
connection.creation.setup_worker_connection(_worker_id)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/django/django/db/backends/sqlite3/creation.py", line 145, in
setup_worker_connection
source_db = self.connection.Database.connect(
f"file:{alias}_{_worker_id}.sqlite3?mode=ro", uri=True
)
sqlite3.OperationalError: unable to open database file
}}}
but you can see that Django successfully created/destroyed the test
databases with the correct names:
{{{#!shell
Cloning test database for alias 'default' ('django.db')...
Cloning test database for alias 'default' ('django.db')...
# test results go here
Destroying test database for alias 'default' ('django_1.db')...
Destroying test database for alias 'default' ('django_2.db')...
Destroying test database for alias 'default' ('django.db')...
}}}
This also affects a multi-database setup if you set the `["TEST"]["NAME"]`
for that database. With the above example, if you run the
`multiple_database` test suite (instead of `model_fields`), you won't see
any errors for the `other` database; as Django will create and connect to
the fallback test name based on the alias, i.e. `other_*.sqlite3`.
However, if you also add `DATABASES["other"]["TEST"] = {"NAME": "other_" +
DATABASE_NAME}` to the settings changes in the above example, the same
error now also happens for the `other` database.
This prevents the use of custom names for running tests with `--parallel`
> 1 (or `auto`), which can be particularly useful when you want to use
`--keepdb` with different names. With `--parallel=1`, this issue does not
occur.
I believe it is because despite the cloning was done correctly using the
given test database file name in
[https://github.com/django/django/blob/2de474dffca008038a7f9fe10289f390c5f4f15b/django/db/backends/sqlite3/creation.py#L58-L60
get_test_db_clone_settings], the logic in
[https://github.com/django/django/blob/2de474dffca008038a7f9fe10289f390c5f4f15b/django/db/backends/sqlite3/creation.py#L145-L147
setup_worker_connection] does not use the clone db names (it always uses
`{alias}_{_worker_id}.sqlite3` instead) when copying them over to the in-
memory database.
FWIW, I encountered this while trying to run
[https://github.com/wagtail/wagtail/pull/13922 Wagtail's test suite] in
parallel with `--keepdb`. I can give a smaller reproducible example if
necessary, but I think the example with Django's own test suite can be a
good example too.
--
Ticket URL: <https://code.djangoproject.com/ticket/36946>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Feb 24, 2026, 10:40:11 AM (2 days ago) Feb 24
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Component: Testing framework | Version: dev
Keywords: tests, sqlite, | Triage Stage:
parallel | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Sage Abdullah):
parallel | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "sqlite_setup_worker_connection_fix.diff" added.
Django
Feb 24, 2026, 10:56:01 AM (2 days ago) Feb 24
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Thanks Sage, I suspected this was not handled but hadn't yet verified.
Would you like to submit a PR?
> the logic in setup_worker_connection does not use the clone db names
(it always uses {alias}_{_worker_id}.sqlite3 instead) when copying them
over to the in-memory database.
Related: I've noticed we have the same `OperationalError` when a parallel
over to the in-memory database.
test worker dies. In that case, the worker_id increments past the number
of databases, and a connection is requested for a nonexistent database. At
a glance, it looks like your suggested patch fixes that as well.
--
Ticket URL: <https://code.djangoproject.com/ticket/36946#comment:1>
Django
Feb 25, 2026, 1:08:24 PM (22 hours ago) Feb 25
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sriniketh99):
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Hi!
I'm interested in working on this ticket.
I'm still learning about Django internals, but I'd like to begin by trying
to reproduce this problem myself and see how the test runner is setting up
the SQLite databases with the --parallel option.
My approach would be to try to understand how the test database names are
being determined and passed down to the worker processes, and then attempt
to solve the problem.
I'll post back here as I make progress and may have questions if I get
stuck.
--
Ticket URL: <https://code.djangoproject.com/ticket/36946#comment:2>
Django
Feb 25, 2026, 1:12:25 PM (22 hours ago) Feb 25
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sage Abdullah):
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Replying to [comment:1 Jacob Walls]:
> Thanks Sage, I suspected this was not handled but hadn't yet verified.
Would you like to submit a PR?
Yep, I'll submit a PR soon. Just trying to find the most appropriate way
Would you like to submit a PR?
to test it.
> Related: I've noticed we have the same `OperationalError` when a
parallel test worker dies. In that case, the worker_id increments past the
number of databases, and a connection is requested for a nonexistent
database. At a glance, it looks like your suggested patch fixes that as
well.
Replying to [comment:2 Sriniketh99]:
> Hi!
> I'm interested in working on this ticket.
> I'm still learning about Django internals, but I'd like to begin by
trying to reproduce this problem myself and see how the test runner is
setting up the SQLite databases with the --parallel option.
> My approach would be to try to understand how the test database names
are being determined and passed down to the worker processes, and then
attempt to solve the problem.
> I'll post back here as I make progress and may have questions if I get
stuck.
Sorry, I should've assigned the ticket to myself. I already made the
> I'm interested in working on this ticket.
> I'm still learning about Django internals, but I'd like to begin by
trying to reproduce this problem myself and see how the test runner is
setting up the SQLite databases with the --parallel option.
> My approach would be to try to understand how the test database names
are being determined and passed down to the worker processes, and then
attempt to solve the problem.
> I'll post back here as I make progress and may have questions if I get
stuck.
patch, so I just need to submit the PR. I'd recommend looking for a
different ticket to work on.
--
Ticket URL: <https://code.djangoproject.com/ticket/36946#comment:3>
Django
Feb 25, 2026, 1:12:39 PM (22 hours ago) Feb 25
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: Sage
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
| Abdullah
Type: Bug | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Sage Abdullah):
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => Sage Abdullah
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/36946#comment:4>
Django
Feb 25, 2026, 3:19:52 PM (20 hours ago) Feb 25
to django-...@googlegroups.com
#36946: Running tests on SQLite with --parallel (using spawn) does not respect
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: Sage
| Abdullah
Type: Bug | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sage Abdullah):
DATABASES["TEST"]["NAME"]
-------------------------------------+-------------------------------------
Reporter: Sage Abdullah | Owner: Sage
| Abdullah
Type: Bug | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: tests, sqlite, | Triage Stage: Accepted
parallel |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
[https://github.com/django/django/pull/20768 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/36946#comment:5>
Reply all
Reply to author
Forward
0 new messages