[Django] #30171: Fix Python warning during LiveServerPort
Django
-------------------------------------+-------------------------------------
Reporter: Jon | Owner: nobody
Dufresne |
Type: | Status: new
Cleanup/optimization |
Component: Database | Version: master
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When running tests with Python warnings enabled, `LiveServerPort` produces
a warning of the form:
{{{
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "django/test/testcases.py", line 1399, in run
connections.close_all()
File "django/db/utils.py", line 224, in close_all
connection.close()
File "django/db/backends/sqlite3/base.py", line 244, in close
self.validate_thread_sharing()
File "django/db/backends/base/base.py", line 531, in
validate_thread_sharing
% (self.alias, self._thread_ident, _thread.get_ident())
django.db.utils.DatabaseError: DatabaseWrapper objects created in a thread
can only be used in that same thread. The object with alias 'default' was
created in thread id 139685002331648 and this is thread id
139684747486976.
}}}
This occurs because multiple classes set
`DatabaseWrapper.allow_thread_sharing` to false upon test tear down. The
base test class and the temporarily created test class. In other words,
nesting the setup/teardown of `DatabaseWrapper.allow_thread_sharing` isn't
always handled cleanly.
--
Ticket URL: <https://code.djangoproject.com/ticket/30171>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/10972 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:1>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* stage: Unreviewed => Accepted
Comment:
Tests aren't passing.
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* version: master => 2.2
* stage: Accepted => Ready for checkin
Old description:
> When running tests with Python warnings enabled, `LiveServerPort`
> produces a warning of the form:
>
> {{{
> Exception in thread Thread-1:
> Traceback (most recent call last):
> File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
> self.run()
> File "django/test/testcases.py", line 1399, in run
> connections.close_all()
> File "django/db/utils.py", line 224, in close_all
> connection.close()
> File "django/db/backends/sqlite3/base.py", line 244, in close
> self.validate_thread_sharing()
> File "django/db/backends/base/base.py", line 531, in
> validate_thread_sharing
> % (self.alias, self._thread_ident, _thread.get_ident())
> django.db.utils.DatabaseError: DatabaseWrapper objects created in a
> thread can only be used in that same thread. The object with alias
> 'default' was created in thread id 139685002331648 and this is thread id
> 139684747486976.
> }}}
>
> This occurs because multiple classes set
> `DatabaseWrapper.allow_thread_sharing` to false upon test tear down. The
> base test class and the temporarily created test class. In other words,
> nesting the setup/teardown of `DatabaseWrapper.allow_thread_sharing`
> isn't always handled cleanly.
New description:
When running tests with Python warnings enabled,
`tests.servers.tests.LiveServerPort` produces a warning of the form:
{{{
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "django/test/testcases.py", line 1399, in run
connections.close_all()
File "django/db/utils.py", line 224, in close_all
connection.close()
File "django/db/backends/sqlite3/base.py", line 244, in close
self.validate_thread_sharing()
File "django/db/backends/base/base.py", line 531, in
validate_thread_sharing
% (self.alias, self._thread_ident, _thread.get_ident())
django.db.utils.DatabaseError: DatabaseWrapper objects created in a thread
can only be used in that same thread. The object with alias 'default' was
created in thread id 139685002331648 and this is thread id
139684747486976.
}}}
This occurs because multiple classes set
`DatabaseWrapper.allow_thread_sharing` to false upon test tear down. The
base test class and the temporarily created test class. In other words,
nesting the setup/teardown of `DatabaseWrapper.allow_thread_sharing` isn't
always handled cleanly.
--
Comment:
The warning appeared after 8c775391b78b2a4a2b57c5e89ed4888f36aada4b, so
we'll backport to stable/2.2.x.
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"76990cbbda5d93fda560c8a5ab019860f7efaab7" 76990cb]:
{{{
#!CommitTicketReference repository=""
revision="76990cbbda5d93fda560c8a5ab019860f7efaab7"
Fixed #30171 -- Fixed DatabaseError in servers tests.
Made DatabaseWrapper thread sharing logic reentrant. Used a reference
counting like scheme to allow nested uses.
The error appeared after 8c775391b78b2a4a2b57c5e89ed4888f36aada4b.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"37cc6a9dce3354cd37f23ee972bc25b0e5cebd5c" 37cc6a9d]:
{{{
#!CommitTicketReference repository=""
revision="37cc6a9dce3354cd37f23ee972bc25b0e5cebd5c"
[2.2.x] Fixed #30171 -- Fixed DatabaseError in servers tests.
Made DatabaseWrapper thread sharing logic reentrant. Used a reference
counting like scheme to allow nested uses.
The error appeared after 8c775391b78b2a4a2b57c5e89ed4888f36aada4b.
Backport of 76990cbbda5d93fda560c8a5ab019860f7efaab7 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chris Jerdonek):
I'm seeing in Django 2.2.17 what looks like the reappearance of #22414
after it being fixed in 1.11. There don't seem to be a whole lot of
changes in the LiveServerTestCase-related code between those two versions
of Django.
I'm wondering if it's possible this commit explains it:
https://github.com/django/django/commit/37cc6a9dce3354cd37f23ee972bc25b0e5cebd5c
For example, was it intentional that this SQLite-specific condition (`if
conn.vendor == 'sqlite' and conn.is_in_memory_db():`) was removed from
`_tearDownClassInternal()`, even though it's still in `setUpClass()`?
Also, for reasons of symmetry with `setUpClass()`, it doesn't seem like
those connection restore lines should have been moved beneath the `if
hasattr(cls, 'server_thread'):` condition. Basically, it looks like a
couple lines were deleted, perhaps by accident.
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson):
Hi Chris.
Can I ask you to open a new issue with a reproduce please? Ideally an
adjustment to the test (such as those modified in
76990cbbda5d93fda560c8a5ab019860f7efaab7) or otherwise a sample project or
similar. Without spending a lot of time, from your comment it's not easy
to see what was missing from the coverage there. Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Jon Dufresne | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chris Jerdonek):
Okay, I created a ticket for the symmetry issue here: #32417. (The
regression issue I created a ticket for here: #32416. I found it wasn't
due to this issue, but to a different one: #20238.)
--
Ticket URL: <https://code.djangoproject.com/ticket/30171#comment:8>