[Django] #29106: Display Skipped Tests in Test Result Output
Django
------------------------------------------------+------------------------
Reporter: Rich Jones | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: 2.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 1 |
------------------------------------------------+------------------------
_This ticket is part of the Django Test Output Beautification initiative:
https://code.djangoproject.com/ticket/29101_
Django 1.10 introduced a wonderful new feature, test tagging, which allows
the user running tests to control which tests are included and excluded in
a test execution. This is particularly handy in complicated and
conditional continuous integration setups, which may require lots of
different tag inclusions and exclusions.
The annoying part is that the default test output, and even the test
output with higher verbosity levels, never displays which tests are
excluded or included.
I'd be great if this information could be displayed in three ways during a
test execution: Once during initialization, to show which tags were
included or excluded, once for each test as it is skipped, and once during
the final tally.
So, at the beginning:
{{{
System check identified no issues (0 silenced).
Running tests, excluding [slow, veryslow].
}}}
For each test skipped:
{{{
========================
Testing 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Makes sure that proper file group runs the pipeline._
-----------------------------------------------------
...
Ok!
========================
Skipping 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
[veryslow]
-----------------------------------------------------
}}}
And finally, at the end:
{{{
----------------------------------------------------------------------
Ran 38 tests in 34.808s. Skipped 4 tests.
}}}
This should work appropriately for all of the different tag-related test
arguments.
It'd also be nice if this test output was appropriately colorized.
Related tickets:
https://code.djangoproject.com/ticket/29105
https://code.djangoproject.com/ticket/22449
https://code.djangoproject.com/ticket/29105
--
Ticket URL: <https://code.djangoproject.com/ticket/29106>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
--------------------------------------+------------------------------------
Reporter: Rich Jones | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: 2.0
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
--------------------------------------+------------------------------------
Changes (by Tim Graham):
* ui_ux: 1 => 0
* stage: Unreviewed => Accepted
Comment:
The tests are filtered by their tag before the test suite starts running,
therefore displaying the skipped tests isn't feasible. Even if it were
feasible, I wouldn't want that in my own use of test tagging. When running
Django's test suite with `--selenium`, for example, there are about 40
tests that run out of hundreds in the complete test suite. Showing all the
skipped tests would just clutter the output for this use case.
Making `DiscoverRunner` displaying the tags seems fine (at verbosity 2 or
greater).
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:1>
Django
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Component: Testing framework | Version: 2.0
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Sanyam Khurana):
* cc: Sanyam Khurana (added)
* owner: nobody => Sanyam Khurana
* status: new => assigned
Comment:
Hey, I'm starting to work on this issue. Will submit a PR soon.
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Sanyam Khurana):
* has_patch: 0 => 1
Comment:
I've added a patch. Wasn't able to write test fully. Need a pointer on how
to proceed. I've mentioned the issue in the PR:
https://github.com/django/django/pull/9658
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):
* needs_tests: 0 => 1
Comment:
I think you'll use `django.test.utils.captured_stdout` for the test.
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Sanyam Khurana):
* needs_tests: 1 => 0
Comment:
Oh, yes. I don't know why I was confused about that.
That did the trick. Thank you so much. I've updated the Pull Request.
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Carlton Gibson):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* stage: Ready for checkin => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Cleanup/optimization | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Sanyam Khurana):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Rich Jones | Owner: Sanyam
Type: | Khurana
Component: Testing framework | Version: 2.0
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"5a669ae2f4e1665f475d384ab762bcf4ddc5660b" 5a669ae2]:
{{{
#!CommitTicketReference repository=""
revision="5a669ae2f4e1665f475d384ab762bcf4ddc5660b"
Fixed #29106 -- Made DiscoverRunner display selected test tags.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29106#comment:9>