[Django] #25209: coverage html doesn't work as documented
Django
-----------------------------------+--------------------
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------
After running the command, under OSX 10.10.4 / Python 3.4.2 (via homebrew)
and coverage==3.7.1:
{{{
coverage run ./runtests.py
}}}
I get no `.coverage` file, after:
https://github.com/django/django/commit/4202959b6febd02cdaa712c666fa0d79569038ca
This appears to be caused by the `parallel = True` option, details:
http://nedbatchelder.com/code/coverage/config.html#h_run
For instance, I currently have:
{{{
(django_py3)pzrq@icebox:~/Projects/django/tests$ ls -a | grep .coverage
.coverage.icebox.local.42652.649933
.coverage.icebox.local.42665.263961
.coverage.icebox.local.42673.382480
...
# Importantly no .coverage file
}}}
This means when you run downstream tools, such as `coverage report -m` or
`coverage html` as recommended in the Django docs, i.e.
https://docs.djangoproject.com/en/1.8/internals/contributing/writing-code
/unit-tests/#code-coverage, you get the not very interesting output like
the following:
{{{
(django_py3)pzrq@icebox:~/Projects/django/tests$ coverage report -m
Name Stmts Miss Cover Missing
-------------------------------------
(django_py3)pzrq@icebox:~/Projects/django/tests$
}}}
I can see two possible fixes to discuss:
1. Remove `parallel = True` from `.coveragerc`
2. Update the documentation such as
https://docs.djangoproject.com/en/dev/internals/contributing/writing-code
/unit-tests/#code-coverage with the intermediate `coverage combine` step,
to merge the different coverage report(s) together to generate a more
typical `.coverage` file.
AFAIK multiprocessing isn't supported by the Django test runner, so +1 for
option 1.
--
Ticket URL: <https://code.djangoproject.com/ticket/25209>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I'm working on a branch that allows parallel execution of Django tests
with `multiprocessing`. It would be nice if it remained compatible with
coverage measurement. Perhaps we can have different instructions for users
of the `--parallel` option once it exists.
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:1>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* cc: MarkusH (added)
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:2>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by pzrq):
@aaugustin
TLDR My PR is currently removing one line. I'll happily put the `coverage
combine` documentation and a revert of that one line removal (i.e.
basically option 2) together tomorrow at PyConAU sprints if you think it
will be useful for your parallel test runner.
But unless that's coming really soon it would be nice to restore the
documentation to a more consistent state as Django does pride itself on
documentation being a first class citizen.
----
https://github.com/django/django/compare/master...aaugustin:parallelize-
tests-attempt-1
That looks really really cool!
I just wonder if how likely it is to make the Django 1.9 alpha around mid
September 2015?
(Using as the guide Django 1.8 which took 2.5 months from alpha to
release, and I fully appreciate that question may be impossible to answer
until 1.9 alpha comes with or without it).
In any case it's a lot grander in scope than my attempt so far (`discover-
road-runner`) - I usually use it dozens of times a day and it works
wonders for my TDD productivity and testing other developers work more
quickly (without needing `--nomigrations` in `django-test-without-
migrations`); but it doesn't handle not-in-memory DBs, live application
servers, file upload, etc. And I'd be the first to admit it's a kludge to
generate test output cleanly and quickly (but has the benefit of not
needing to pickle things like tracebacks if you print them straight away),
and handle the really expensive migrations step better. With any luck OSX
10.11 El Capitan will mean you won't have to worry about all the headaches
of the builtin `multiprocessing` module under OSX Python 2.7.6 at least (a
colleague suggested I use the `billiard` fork entirely due to the time I
have spent wrangling `multiprocessing` and `sqlite3` crashes which AFAIK
are fixed upstream and certainly in Python 3.4). I guess this is just a
longer way of saying good luck - parallelism can be very challenging and
fun!
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:3>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Accepted => Ready for checkin
Comment:
I think it's fine to commit this for now. Markus's project to improve
coverage on Jenkins hasn't been completed and it's not clear to me that
`parallel=True` should be the default..
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:4>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by shaib):
* cc: shaib (added)
Comment:
FWIW, I promised to get Oracle support for the parallel test runner, and
then couldn't really complete the work because of RL etc. I'll have more
time for this after this weekend (when a conference I'm organizing is
over).
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:5>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
I admit I might was a bit too eager to get parallel coverage support on
Jenkins running but staled due to a missing feature in coverage:
https://bitbucket.org/ned/coveragepy/issues/277/combined-parallel-report-
should-use
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:6>
Django
Reporter: pzrq | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"f6259ce776eb022d1ca05fc5f4e5bc56ca1a4728" f6259ce7]:
{{{
#!CommitTicketReference repository=""
revision="f6259ce776eb022d1ca05fc5f4e5bc56ca1a4728"
Fixed #25209 -- Removed parallel=True coverage option
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25209#comment:7>