Re: [Django] #36770: SQLite threading tests are flaky when parallel test suite runs in forkserver mode

[Django]

#36770: SQLite threading tests are flaky when parallel test suite runs in
forkserver mode
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Kundan
Type: | Yadav
Cleanup/optimization | Status: assigned
Component: Testing framework | Version: 5.2
Severity: Normal | Resolution:
Keywords: 3.14, forkserver, | Triage Stage: Accepted
parallel |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jacob Walls):

I've seen this intermittently locally. Leaving aside the assertion
failures for SQLite, we shouldn't have a hang in `LiveServerTestCase` when
tests fail. You can engineer a hang like this, setting a miniscule timeout
that will always raise:

{{{#!py
diff --git a/django/test/testcases.py b/django/test/testcases.py
index 5f83612fe5..622b938dd6 100644
--- a/django/test/testcases.py
+++ b/django/test/testcases.py
@@ -1844,7 +1844,8 @@ class LiveServerTestCase(TransactionTestCase):
cls.addClassCleanup(cls._terminate_thread)

# Wait for the live server to be ready
- cls.server_thread.is_ready.wait()
+ if not cls.server_thread.is_ready.wait(timeout=0.001):
+ raise Exception("Live server never became ready.")
if cls.server_thread.error:
raise cls.server_thread.error
}}}


Then when `KeyboardInterrupt`ing out of it, you get a stack trace from
`doClassCleanups`, suggesting that the termination code is waiting
forever, even though the live server never started:

{{{#!py
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/suite.py",
line 181, in _handleClassSetUp
doClassCleanups()
~~~~~~~~~~~~~~~^^
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py",
line 720, in doClassCleanups
function(*args, **kwargs)
~~~~~~~~^^^^^^^^^^^^^^^^^
File "/Users/jwalls/django/django/test/testcases.py", line 1864, in
_terminate_thread
cls.server_thread.terminate()
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Users/jwalls/django/django/test/testcases.py", line 1788, in
terminate
self.join()
}}}

Something like this fixes it:

{{{#!diff
diff --git a/django/test/testcases.py b/django/test/testcases.py
index 5f83612fe5..9cbeeeca25 100644
--- a/django/test/testcases.py
+++ b/django/test/testcases.py
@@ -1781,11 +1781,12 @@ class LiveServerThread(threading.Thread):
)

def terminate(self):
- if hasattr(self, "httpd"):
- # Stop the WSGI server
- self.httpd.shutdown()
- self.httpd.server_close()
- self.join()
+ if self.is_ready.is_set():
+ if hasattr(self, "httpd"):
+ # Stop the WSGI server
+ self.httpd.shutdown()
+ self.httpd.server_close()
+ self.join()
}}}

My theory is that the "live server never became ready" situation I
simulated above is similar to the situation we're seeing on CI where a
database lock entails a failure to start a live server thread.
----
Then for one of the underlying assertion failures, I don't know how I feel
about masking a real problem, but we could probably reduce the chance of
failing jobs by adjusting `test_in_memory_database_lock()` to use the
`other` database instead of the `default`. It would still cover the code
it's testing, but it would just have a much smaller chance of interacting
poorly with other tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/36770#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.