[Django] #33149: manage.py test --pdb does not open a debugger when a subtest fails
Django
---------------------------------------------+------------------------
Reporter: Lucidiot | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 3.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
With a test case that contains a `TestCase.subTest` call, a failure or
error of a subtest does not cause pdb or ipdb to open when running
`manage.py test --pdb`.
{{{
#!python
from django.test import TestCase
class TestSomething(TestCase):
def test_something(self):
with self.subTest(a=1):
raise ZeroDivisionError("oh snap!")
}}}
{{{
λ ./manage.py test --pdb
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
...
======================================================================
ERROR: test_something (demo.app.tests.test_something.TestSomething) (a=1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/.../tests/test_something.py", line 6, in test_something
raise ZeroDivisionError("oh snap!")
ZeroDivisionError: oh snap!
----------------------------------------------------------------------
Ran 1 tests in 0.154s
FAILED (errors=1)
Destroying test database for alias 'default'...
}}}
It seems that CPython has some
[https://github.com/python/cpython/blob/b0a6ede3d0bd6fa4ffe413ab4dfc1059201df25b/Lib/unittest/runner.py#L73
special handling code for subtest results] that does not use
`TestResult.addError` or `TestResult.addFailure`, and the `PDBDebugResult`
[https://github.com/django/django/blob/b263f4b69db4093847ccc3b85e51cc7f3759e42c/django/test/runner.py#L102-L108
only implements those]. Note that `--debug-sql` does work with subtests
because the `DebugSQLTextTestResult`
[https://github.com/django/django/blob/b263f4b69db4093847ccc3b85e51cc7f3759e42c/django/test/runner.py#L79-L84
supports subtests].
--
Ticket URL: <https://code.djangoproject.com/ticket/33149>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 3.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* stage: Unreviewed => Accepted
Comment:
Thanks for the report. It seems it's enough to call `debug()` 🤔, e.g.
{{{
def addSubTest(self, test, subtest, err):
if err is not None:
self.debug(err)
super().addSubTest(test, subtest, err)
}}}
Would you like to provide a patch?
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:1>
Django
-----------------------------------+------------------------------------
Type: Bug | Status: assigned
Component: Testing framework | Version: 3.2
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Abhyudai):
* owner: nobody => Abhyudai
* status: new => assigned
Comment:
for what it is worth, adding the call to `debug()` didn't work, at least
not on my end. i will try to look more into this.
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:2>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Lucidiot):
Sorry for the delay! Adding a `self.debug` call as in the code block above
does seem to work on my machine, at least when hastily adding it by
editing my `site-packages` (don't try this at home). I could maybe provide
a patch, and Abhyudai can test it to see if something is truly missing?
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:3>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Abhyudai):
Just to be on the same page, this is the diff that is being talked about,
no?
{{{
diff --git a/django/test/runner.py b/django/test/runner.py
index 225bc19b09..9ea67ce4b6 100644
--- a/django/test/runner.py
+++ b/django/test/runner.py
@@ -275,6 +275,7 @@ failure and get a correct traceback.
self.check_picklable(test, err)
self.check_subtest_picklable(test, subtest)
self.events.append(('addSubTest', self.test_index, subtest,
err))
+ self.debug(err)
super().addSubTest(test, subtest, err)
def addSuccess(self, test):
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:4>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Lucidiot):
No, I added the method to the PDBDebugResult directly:
{{{
diff --git a/django/test/runner.py b/django/test/runner.py
index 225bc19b09..b4dd974d57 100644
--- a/django/test/runner.py
+++ b/django/test/runner.py
@@ -107,6 +107,10 @@ class PDBDebugResult(unittest.TextTestResult):
super().addFailure(test, err)
self.debug(err)
+ def addSubTest(self, test, subtest, err):
+ super().addSubTest(test, subtest, err)
+ self.debug(err)
+
def debug(self, error):
self._restoreStdout()
self.buffer = False
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:5>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Abhyudai):
* has_patch: 0 => 1
Comment:
yes, it seems to work. thanks
[https://github.com/django/django/pull/14962/ pull-request]
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:6>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Component: Testing framework | Version: 3.2
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by GitHub <noreply@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"7872971dfbb818177168e64c24a933e48ce01206" 7872971d]:
{{{
#!CommitTicketReference repository=""
revision="7872971dfbb818177168e64c24a933e48ce01206"
Fixed #33149 -- Made test runner --pdb option work with subTest().
Thanks Lucidot for the report and Mariusz Felisiak for the initial
patch.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:7>
Django
-----------------------------------+------------------------------------
Reporter: Lucidiot | Owner: Abhyudai
Type: Bug | Status: closed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"44d11e5b2c6d61916ff5d7e9549f6c6911f5044d" 44d11e5b]:
{{{
#!CommitTicketReference repository=""
revision="44d11e5b2c6d61916ff5d7e9549f6c6911f5044d"
[4.0.x] Fixed #33149 -- Made test runner --pdb option work with subTest().
Thanks Lucidot for the report and Mariusz Felisiak for the initial
patch.
Backport of 7872971dfbb818177168e64c24a933e48ce01206 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33149#comment:8>