[Django] #27391: Support unittest.TestCase.debug() method
Django
-------------------------------+-------------------------------------------
Reporter: Pavel | Owner: nobody
Savchenko |
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.10
Severity: Normal | Keywords: unittest SimpleTestCase debug
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-------------------------------------------
== Introduction ==
The documentation for
[https://docs.python.org/2/library/unittest.html#unittest.TestCase.debug
unittest.TestCase.debug] states that invoking it would:
Run the test without collecting the result. This allows exceptions
raised by the test to be propagated to the caller, and can be used to
support running tests under a debugger.
In essence, `debug()` invokes the test method same as
[https://docs.python.org/2/library/unittest.html#unittest.TestCase.run
TestCase.run()] does, without the added functionality of catching
exceptions and related cruft.
== Side effects ==
A recent change in pytest (see references below) brought to our attention
the fact that Django does not implement a `debug` method, and performs
important stuff in `TestCase.__call__` (besides calling `run()`), such as
calling `_pre_startup` and `_post_teardown` methods.
Thus, since debug() is unimplemented, running debug on a django TestCase
(for example for debugging purposes), results unintended behavior, such as
certain methods not being called. Most importantly `_fixture_callback` and
`_fixture_teardown` which also perform database rollback (or call
`flush`).
References:
https://github.com/pytest-dev/pytest/pull/1890
https://github.com/pytest-dev/pytest/issues/1932
https://github.com/pytest-dev/pytest-django/pull/406
https://github.com/python/cpython/blob/master/Lib/unittest/case.py#L651-L658
--
Ticket URL: <https://code.djangoproject.com/ticket/27391>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage:
SimpleTestCase debug | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* component: Uncategorized => Testing framework
* needs_better_patch: => 0
* type: Uncategorized => New feature
* needs_tests: => 0
* needs_docs: => 0
Comment:
Would this have benefit outside of the usage of third-party test runners
such as `pytest`? If so, what does it look like?
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage:
SimpleTestCase debug | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Pavel Savchenko):
Well, not that I know of yet, but basically, any test runner that allows
unittests may assume "debug" can be used for exception debugging; and
right now this cannot be done without issues when using Django's
SimpleTestCase derivatives.
Regardless, the patch should be fairly simple 10 line method and I can
submit that. It's invoking the test that I'm having trouble with, how to
invoke "debug" as part of the regular test suite...
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
SimpleTestCase debug |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Pavel Savchenko):
As for other test runners, here's now nose2 invokes a debugger (IIRC
pytest does something similar):
https://github.com/nose-
devs/nose2/blob/master/nose2/plugins/debugger.py#L41-L60
This means that the `post_mortem` is invoked only after the exception was
already caught in `SimpleTestCase.__call__` and, for example, the database
was already rolled back. The test frameworks simple have no real way to
examine the state of the application at the time of the error (but only
"after", when it may be too late).
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Pavel Savchenko):
* has_patch: 0 => 1
Comment:
Added test case (demonstrating usage expectations) and resolution in
[https://github.com/django/django/pull/7436 PR 7436]
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Adam Chainz):
* cc: me@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* needs_better_patch: 0 => 1
Comment:
I think the test could be a bit cleaner. I sent some edits to the PR,
though I broke some things too.
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: 1.10
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Martin):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Needs tests: 0 | Patch needs improvement: 1
Changes (by Asif Saifuddin Auvi):
* needs_docs: 0 => 1
* version: 1.10 => master
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Pavel Savchenko):
* needs_better_patch: 1 => 0
Comment:
Replying to [comment:10 Asif Saifuddin Auvi]:
Patch was improved as per Tim Martin's notes
[https://github.com/django/django/pull/7436/commits/7ec5fae434f01ebef77e2d7bb2544bf5314d99bf
in commit 7ec5fae]. I added a sketch for release note mention in
[https://github.com/django/django/pull/7436#issuecomment-350575023 this
comment].
Is there other documentation you had in mind?
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: unittest | Triage Stage: Accepted
SimpleTestCase debug |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"dc8cd2fefd028913e3273927342374f5259cc566" dc8cd2fe]:
{{{
#!CommitTicketReference repository=""
revision="dc8cd2fefd028913e3273927342374f5259cc566"
Refs #27391 -- Added more tests for SimpleTestCase.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
SimpleTestCase debug | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: Pavel Savchenko | Owner: nobody
Component: Testing framework | Version: master
Keywords: unittest | Triage Stage: Ready for
SimpleTestCase debug | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"1711c509faf3111bdf5a3a860b2cd01c0dc5d233" 1711c509]:
{{{
#!CommitTicketReference repository=""
revision="1711c509faf3111bdf5a3a860b2cd01c0dc5d233"
Fixed #27391 -- Implemented SimpleTestCase.debug().
debug() should bubbled up exceptions if occurring in test, but behave
the same as run() when no exceptions occurred.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27391#comment:17>