[Django] #34210: Show the duration of individual tests
Django
-------------------------------------+-------------------------------------
Reporter: Paolo | Owner: nobody
Melchiorre |
Type: New | Status: new
feature |
Component: Testing | Version: dev
framework | Keywords: test, timing,
Severity: Normal | durations
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I propose to add the possibility to show the duration of individual tests
in the final report by ordering them from slowest to fastest and filtering
only the slowest tests with a threshold.
Example:
{{{
$ python -m manage test --durations=10
Found 92 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
Creating test database for alias 'default'...
..........
----------------------------------------------------------------------
Ran 92 tests in 1.297s
OK
Destroying test database for alias 'default'...
Slowest 10 test duration durations:
0.3597s test_detail_view_with_a_future_poll
(polls.tests.PollIndexDetailTests)
0.0284s test_detail_view_with_a_past_poll
(polls.tests.PollIndexDetailTests)
0.0068s test_index_view_with_a_future_poll (polls.tests.PollViewTests)
0.0047s test_index_view_with_a_past_poll (polls.tests.PollViewTests)
0.0045s test_index_view_with_two_past_polls (polls.tests.PollViewTests)
0.0041s test_index_view_with_future_poll_and_past_poll
(polls.tests.PollViewTests)
0.0036s test_index_view_with_no_polls (polls.tests.PollViewTests)
0.0003s test_was_published_recently_with_future_poll
(polls.tests.PollMethodTests)
0.0002s test_was_published_recently_with_recent_poll
(polls.tests.PollMethodTests)
0.0002s test_was_published_recently_with_old_poll
(polls.tests.PollMethodTests)
}}}
`pytest` has a very similar option that helps you find slower tests that
you want to optimize their timing.
https://pytest.org/en/latest/how-to/usage.html#durations
There's an external package `django-slow-tests` that adds a similar
functionality, but the code is very old and with very little test
coverage, furthermore the last release was almost 4 years ago and the
officially supported versions of Django are very old.
https://github.com/realpython/django-slow-tests
I propose to add this feature directly into Django tests to help users
optimize their tests and also to help Django developers optimize the test
of Dajngo itself.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Adam Johnson (added)
* stage: Unreviewed => Accepted
Comment:
I think this would be great. (I'm sure Adam mentioned similar once upon a
timeβ¦ π€)
I could do with the same for Django's own test suite β there's one test
that I notice sitting there, which would be good to get a bead on π
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Paolo Melchiorre):
Replying to [comment:1 Carlton Gibson]:
> I think this would be great. (I'm sure Adam mentioned similar once upon
a timeβ¦ π€)
Sure, Adam talked about the two alternatives in his book.
Ever since I read this, I started wondering if it might not have been a
good idea to have this function directly in Django.
> I could do with the same for Django's own test suite β there's one test
that I notice sitting there, which would be good to get a bead on π
By often running Django tests locally, lately I too have noticed that for
a particular test the tests remain pending for a few seconds and it would
not hurt to find out which one it is and optimize it.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
As far as I'm aware, we declined to add `--durations` in this
[https://github.com/django/django/pull/13224 PR].
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson):
I think last comment there was mine here
https://github.com/django/django/pull/13224#issuecomment-664913219 β pro
durations but handling it separately, so here maybe :)
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Adam Johnson):
django-rich has an open PR from David Smith to add test timing
functionality: https://github.com/adamchainz/django-rich/pull/30
Itβs stalled because of sorting out parallel testing, and trying to also
time setUp / tearDown / setUpClass / tearDownClass.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Paolo Melchiorre):
Seems to me like a very useful feature to have right in Django tests.
It would be very useful to be able to use this feature also to optimize
Django's tests themselves.
It could be a feature that a tool like Django Rich could add color to?
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Adam Johnson):
Yes, though the initial idea was to implement it first in a third party
package before proposing it for Django itself.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Paolo Melchiorre):
I think it's a good idea.
At the same time I thought it would be interesting to take advantage of
this functionality to discover w and slow tests of Django itself, and I
don't think it can be done using an external package.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:8>
Django
-------------------------------------+-------------------------------------
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => shaswat10
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: shaswat
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: David Smith (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: shaswat
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Ruchir Harbhajanka (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: shaswat
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by David Smith):
Python 3.12 will add a --durations flag.
https://github.com/python/cpython/pull/12271
I am leaning towards saying thats enough to call this wontfix.
While it's likely that a bit more investigation is needed to see if it
covers SetUp / TearDown I wonder if its reasonable to say let's implement
that additional functionality in a 3rd party package first?
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: shaswat
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Adam Johnson):
That sounds good. Even with the 3.12 support, Django will still need to
forward the option through to unittest.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: shaswat
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Adam Johnson):
Oh, and the unittest flag will not work with parallel testing - there will
need to be some work in Django to merge the test times across processes.
For future reference David opened a PR in django-rich for timing:
https://github.com/adamchainz/django-rich/pull/30 . It stalled due to my
prefectionism trying to time all setups and teardowns...
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:14>
Django
-------------------------------------+-------------------------------------
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: shaswat => David Smith
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by David Smith):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce):
I think this also will close this ticket:
https://code.djangoproject.com/ticket/31827
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
Replying to [comment:17 Sarah Boyce]:
> I think this also will close this ticket:
https://code.djangoproject.com/ticket/31827
Probably yes, let's wait for the final release first.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:18>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
#31827 was a duplicate for showing N slowest tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:19>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: test, timing, | Triage Stage: Accepted
durations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
Comment:
Setting the flag "patch needs improvement" to remove this work from the
review queue (and also because the branch will need a rebase and conflict
fixes before landing).
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:20>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
durations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:21>
Django
-------------------------------------+-------------------------------------
Reporter: Paolo Melchiorre | Owner: David
| Smith
Component: Testing framework | Version: dev
Keywords: test, timing, | Triage Stage: Ready for
durations | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"74b5074174d1749ee44df2f7ed418010a7a4ac70" 74b50741]:
{{{
#!CommitTicketReference repository=""
revision="74b5074174d1749ee44df2f7ed418010a7a4ac70"
Fixed #34210 -- Added unittest's durations option to the test runner.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34210#comment:22>