[Django] #29105: Make Default Test Output More Verbose (And Formatted Prettier)
Django
------------------------------------------------+------------------------
Reporter: Rich Jones | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: 2.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 1 |
------------------------------------------------+------------------------
''This ticket is part of the Django Test Output Beautification initiative:
https://code.djangoproject.com/ticket/29101''
The current default test output provides no information about what test is
currently running. This makes the test output pretty useless if you're
actually trying to determine which test is taking so long, or failing, or
misbehaving, etc.
The current test output is a wall of unformatted text - bleh! With
--verbosity=3, it includes information about which test is running, and
the description of the test (yay), but the formatting is still horrible.
{{{
test_bad_file_grouping
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
Raises exception if both files don't have the same download_url. ...
2018-02-02 15:31:50,587 local/MainProcess
data_refinery_workers.downloaders.transcriptome_index ERROR
[downloader_job: 12]: A Batch's file doesn't have the same download URL as
the other batch's file.
ok
test_good_file_grouping
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
Raises exception if both files don't have the same download_url. ...
ok
}}}
We should make the _default_ test output to be something like:
{{{
========================
Testing 'test_bad_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Raises exception if both files don't have the same download_url._
-----------------------------------------------------
...
ERROR [downloader_job: 12]: A Batch's file doesn't have the same download
URL as the other batch's file.
Ok!
========================
Testing 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Makes sure that proper file group runs the pipeline._
-----------------------------------------------------
...
Ok!
}}}
Where test title, test description and Ok/Failure are also colorized
appropriately.
I'm not totally certain what else the 'verbosity' argument is controlling,
(perhaps log level?), but either way I think the way to improve this
experience is clear: Move to a more structured, human-friendly text
formatting and away from barfing out a wall of unformatted text.
Related tickets:
https://code.djangoproject.com/ticket/29101
https://code.djangoproject.com/ticket/22449
--
Ticket URL: <https://code.djangoproject.com/ticket/29105>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Rich Jones | Owner: nobody
Cleanup/optimization |
Component: Testing framework | Version: 2.0
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 1
Comment (by Chris Jerdonek):
I like that the default output prints just a single `.` for each test. For
example, if you have 1000 tests, you wouldn't want 5 lines per test, or
even one line.
However, I agree the format can be improved when the verbosity is
increased. I do think one of the available verbosity levels should show
the "minimal" information needed to know what test is being run -- so
without any divider lines, test description, etc. It can be as simple as a
single line that looks something like:
{{{
Testing: <test name>
}}}
or perhaps prefixed with the number of the test:
{{{
1. Testing: <test name>
}}}
Higher verbosity levels can show more like the description, divider lines,
etc.
--
Ticket URL: <https://code.djangoproject.com/ticket/29105#comment:1>
Django
Reporter: Rich Jones | Owner: nobody
Cleanup/optimization |
Component: Testing framework | Version: 2.0
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 1
Changes (by Tim Graham):
* status: new => closed
* resolution: => wontfix
Comment:
Django is delegating to Python's testing output here. I'm not immediately
convinced that we should implement our own test output. As I mentioned in
the other tickets, there are a number of third-party testing tools that
offer other outputs. Do we need to spend our time doing something similar?
Feel free to start a thread on the DevelopersMailingList to determine if
there's a consensus.
--
Ticket URL: <https://code.djangoproject.com/ticket/29105#comment:2>