[Django] #20700: Tests does not report missing fixtures
Django
-----------------------------------+--------------------
Reporter: vzima | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------
If a test contains fixtures, which do not exists, it does not report it as
as failure. This is serious problem as fixtures are part of the test and
their absence may cause false-positive success.
Consider this example:
{{{
#!python
from django.test import TestCase
class TestFoo(TestCase):
fixtures = ('/invalid/filename', ) # Assume the fixture should load
FooModel(pk=1)
def test_foo_delete(self):
delete_if_foo_exists(pk=1)
self.assertFalse(FooModel.objects.filter(pk=1).exists())
}}}
Such a test would pass even if the `delete_if_foo_exists` does nothing.
This test is a very simple example, but ignoring the fact that fixtures
are not loaded, can easily hide less obvious and more severe errors. Mere
typo in fixture name can very well lead to a serious error.
--
Ticket URL: <https://code.djangoproject.com/ticket/20700>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: vzima | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 1.5
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by jacobh):
* status: new => closed
* needs_better_patch: => 0
* resolution: => duplicate
* needs_tests: => 0
* needs_docs: => 0
Comment:
Duplicate of #10200
--
Ticket URL: <https://code.djangoproject.com/ticket/20700#comment:1>
Django
Reporter: vzima | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by vzima):
* version: 1.5 => master
Comment:
#10200 did not fixed this error. Tested on master.
{{{
#!python
class TestFoo(unittest.TestCase):
fixtures = ('/invalid/filename', )
def test_foo(self):
self.fail('Should not run')
}}}
The main problem here is the `loaddata` does not report any error when
file is missing.
--
Ticket URL: <https://code.djangoproject.com/ticket/20700#comment:2>
Django
Reporter: vzima | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by kmtracey):
* status: closed => new
* resolution: duplicate =>
Comment:
Reopen for further investigation given comment that the fix for noted dupe
ticket does not fix this particular problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/20700#comment:3>
Django
Reporter: vzima | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by ramiro):
See also #18990 and
https://github.com/django/django/commit/7a99d1e167f81c5fcd1b9f7f548c5d6959cef2ef
--
Ticket URL: <https://code.djangoproject.com/ticket/20700#comment:4>
Django
Reporter: vzima | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by kmtracey):
* status: new => closed
* resolution: => duplicate
Comment:
OK so you now do get a warning when running a test such as the one in the
original description:
{{{
(django2) kmtracey@caktus006 10:26:34: ~/software/web/playground
--> ./manage.py test
Creating test database for alias 'default'...
/home/kmtracey/django/git-
django/django/core/management/commands/loaddata.py:218: UserWarning: No
fixture named 'filename' found.
warnings.warn("No fixture named '%s' found." % fixture_name)
.
----------------------------------------------------------------------
Ran 1 test in 0.004s
OK
Destroying test database for alias 'default'...
}}}
That seems sufficient to me, even though the test does still "pass", I
don't think it's too much to expect that devs will note the warning and
investigate. So re-closing this as a dupe of #18990.
--
Ticket URL: <https://code.djangoproject.com/ticket/20700#comment:5>