[Django] #30413: tests: admin_views.test_multidb fails with persistent sqlite: OperationalError: database is locked
Django
OperationalError: database is locked
---------------------------------------------+------------------------
Reporter: Daniel Hahler | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
I've tried using persistent SQLite databases for the tests (to make use of
`--keepdb`), but at least some test fails with:
> sqlite3.OperationalError: database is locked
This is not an issue when only using `TEST["NAME"]` with "default" (which
is good enough in terms of performance).
{{{
diff --git i/tests/test_sqlite.py w/tests/test_sqlite.py
index f1b65f7d01..9ce4e32e14 100644
--- i/tests/test_sqlite.py
+++ w/tests/test_sqlite.py
@@ -15,9 +15,15 @@
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
+ 'TEST': {
+ 'NAME': 'test_default.sqlite3'
+ },
},
'other': {
'ENGINE': 'django.db.backends.sqlite3',
+ 'TEST': {
+ 'NAME': 'test_other.sqlite3'
+ },
}
}
}}}
{{{
% tests/runtests.py admin_views.test_multidb -v 3 --keepdb --parallel 1
…
Operations to perform:
Synchronize unmigrated apps: admin_views, auth, contenttypes, messages,
sessions, staticfiles
Apply all migrations: admin, sites
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application sites
Running pre-migrate handlers for application sessions
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application admin_views
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
No migrations to apply.
Running post-migrate handlers for application contenttypes
Running post-migrate handlers for application auth
Running post-migrate handlers for application sites
Running post-migrate handlers for application sessions
Running post-migrate handlers for application admin
Running post-migrate handlers for application admin_views
System check identified no issues (0 silenced).
ERROR
======================================================================
ERROR: setUpClass (admin_views.test_multidb.MultiDatabaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "…/Vcs/django/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "…/Vcs/django/django/db/backends/sqlite3/base.py", line 391, in
execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "…/Vcs/django/django/test/testcases.py", line 1137, in setUpClass
cls.setUpTestData()
File "…/Vcs/django/tests/admin_views/test_multidb.py", line 40, in
setUpTestData
username='admin', password='something', email='te...@test.org',
File "…/Vcs/django/django/contrib/auth/models.py", line 158, in
create_superuser
return self._create_user(username, email, password, **extra_fields)
File "…/Vcs/django/django/contrib/auth/models.py", line 141, in
_create_user
user.save(using=self._db)
File "…/Vcs/django/django/contrib/auth/base_user.py", line 66, in save
super().save(*args, **kwargs)
File "…/Vcs/django/django/db/models/base.py", line 741, in save
force_update=force_update, update_fields=update_fields)
File "…/Vcs/django/django/db/models/base.py", line 779, in save_base
force_update, using, update_fields,
File "…/Vcs/django/django/db/models/base.py", line 870, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk,
raw)
File "…/Vcs/django/django/db/models/base.py", line 908, in _do_insert
using=using, raw=raw)
File "…/Vcs/django/django/db/models/manager.py", line 82, in
manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "…/Vcs/django/django/db/models/query.py", line 1175, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "…/Vcs/django/django/db/models/sql/compiler.py", line 1321, in
execute_sql
cursor.execute(sql, params)
File "…/Vcs/django/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "…/Vcs/django/django/db/backends/utils.py", line 76, in
_execute_with_wrappers
return executor(sql, params, many, context)
File "…/Vcs/django/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "…/Vcs/django/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "…/Vcs/django/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "…/Vcs/django/django/db/backends/sqlite3/base.py", line 391, in
execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database is locked
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30413>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
OperationalError: database is locked
Reporter: Daniel Hahler | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Daniel Hahler):
This is only an issue when setting `TEST["NAME"]`, but not `NAME`.
The following works:
{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'django_tests_default.sqlite3',
},
'other': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'django_tests_other.sqlite3',
}
}
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:1>
Django
-----------------------------------+------------------------------------
Reporter: Daniel Hahler | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* version: 2.2 => master
* stage: Unreviewed => Accepted
Comment:
Reproduced at 0dd2308cf6f559a4f4b50edd7c005c7cf025d1aa.
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:2>
Django
Reporter: Daniel Hahler | Owner: Ngalim
| Siregar
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ngalim Siregar):
* owner: nobody => Ngalim Siregar
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Ngalim
| Siregar
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ngalim Siregar):
* has_patch: 0 => 1
Old description:
New description:
I've tried using persistent SQLite databases for the tests (to make use of
`--keepdb`), but at least some
tsbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
+ export PATHest fails with:
> sqlite3.OperationalError: database is locked
--
Comment:
Created [https://github.com/django/django/pull/11678 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Ngalim
| Siregar
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by felixxm:
Old description:
> I've tried using persistent SQLite databases for the tests (to make use
> of
> `--keepdb`), but at least some
> tsbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
> + export PATHest fails with:
New description:
> sqlite3.OperationalError: database is locked
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Ngalim
| Siregar
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:6>
Django
-------------------------------------+-------------------------------------
| Bukhsh
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Farhaan Bukhsh):
* owner: Ngalim Siregar => Farhaan Bukhsh
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Farhaan
| Bukhsh
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Comment (by Farhaan Bukhsh):
Hey, I am able to replicate this bug and was able to fix it as well with
the help of https://github.com/django/django/pull/11678, but the point I
am stuck at is how to test it,
I am not able to manipulate the `cls` variable so the next option that is
left is create a file like `test_sqlite` and pass it as a parameter in
`runtests`, should I be doing that?
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Farhaan
| Bukhsh
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Comment (by felixxm):
I think we should add `tests/backends/sqlite/test_creation.py` with
regressions tests for `test_db_signature()`, you can take a look at
`tests/backends/base/test_creation.py` with similar tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Farhaan
| Bukhsh
Type: Bug | Status: assigned
Component: Testing framework | Version: master
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Daniel Hahler | Owner: Farhaan
| Bukhsh
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"664c98f1f8c63a932a9c767a7364a6a1826799bb" 664c98f1]:
{{{
#!CommitTicketReference repository=""
revision="664c98f1f8c63a932a9c767a7364a6a1826799bb"
Fixed #30413 -- Fixed test database signature on SQLite when test database
name is provided.
Previously, the same signature was created for multiple in-memory
databases on SQLite when they had tests databases names
DATABASES['TEST']['NAME'].
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30413#comment:11>