Testing in Django - newbie questions on first test failure and coverage

[Derek]

I have an existing Django (1.4) project, with multiple apps and extensive business logic, that I need to write tests for.  Based on a day or two of reading of the core Django docs and numerous blogs (each of which go about things subtly differently!?), I have made a start.  As part of the seeming "best practice" setup, I have also installed django-nose and coverage.

Currently, I have problems with my first test not working and also with coverage seemingly not finding my test.
 
I have a separate `tests` directory in my project, with sub-directories; each corresponding to an app.  Each sub-directory then has a models, views, and functions Python files; acting as placeholders for the test code I think need to write.  In the root of the `tests` directory, I have an __init__.py file that has a number of lines that look like `from myproj.app1.functions import *`.

I have changed the settings.py file to look like this (so that the tests use a "fast" sqlite database):

if 'test' in sys.argv:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': 'test_db'
        }
    }
else:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            # etc ... normal setup


The first test, in the first functions.py file, looks like this:


from django.test import TestCase
from django.core import management
from app1.models import default_Property  # function to be tested

def setup():
    management.call_command('loaddata', 'app1/fixtures/initial_data.json', verbosity=1)

def teardown():
    management.call_command('flush', verbosity=1, interactive=False)

class FunctionsTestCase(TestCase):

    def _fixture_setup(self):
        pass

    def test_default_Property(self):
        self.assertEqual(default_Property(), None)


The default_Property() function in app1 is just set to `return None` for now, so that the above test should work.

However, the test fails.  I get this strange error:

======================================================================
ERROR: test_default_Property (myproj.app1.tests.FunctionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 508, in __call__
    self._post_teardown()
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 522, in _post_teardown
    self._fixture_teardown()
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 847, in _fixture_teardown
    transaction.leave_transaction_management(using=db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 52, in leave_transaction_management
    connection.leave_transaction_management()
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", line 115, in leave_transaction_management
    raise TransactionManagementError("This code isn't under transaction "
TransactionManagementError: This code isn't under transaction management
----------------------------------------------------------------------

I could not find an obvious solution to this, as I do not even have TransactionMiddleware enabled in my settings.

The other issue relates to coverage - it does not seem to recognise I have written this test and still flags it as `red` ... Currently I am using `coverage html' - but how I get it work properly?

Any help getting started overcoming these issues will be helpful, so I can get on with the actual business of writing tests.  (As a side-note, if there is a good existing code base, for a Django-based project, that has test code I could look at, I'd appreciate a link to it.)

Thanks
Derek

[Rafael Durán Castañeda]

Hi,

Some answers inline and links at the end:

2013/9/15 Derek <game...@gmail.com>

I have an existing Django (1.4) project, with multiple apps and extensive business logic, that I need to write tests for.  Based on a day or two of reading of the core Django docs and numerous blogs (each of which go about things subtly differently!?), I have made a start.  As part of the seeming "best practice" setup, I have also installed django-nose and coverage.

Currently, I have problems with my first test not working and also with coverage seemingly not finding my test.
 
I have a separate `tests` directory in my project, with sub-directories; each corresponding to an app.  Each sub-directory then has a models, views, and functions Python files; acting as placeholders for the test code I think need to write.  In the root of the `tests` directory, I have an __init__.py file that has a number of lines that look like `from myproj.app1.functions import *`.

 

Most Django apps. place tests into the app. directory tests module/package, so I think your layout may cause some Django test runner may not able to find your tests. 

I have changed the settings.py file to look like this (so that the tests use a "fast" sqlite database):

if 'test' in sys.argv:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': 'test_db'
        }
    }
else:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            # etc ... normal setup


The first test, in the first functions.py file, looks like this:


from django.test import TestCase
from django.core import management
from app1.models import default_Property  # function to be tested

def setup():
    management.call_command('loaddata', 'app1/fixtures/initial_data.json', verbosity=1)

def teardown():
    management.call_command('flush', verbosity=1, interactive=False)

class FunctionsTestCase(TestCase):

    def _fixture_setup(self):
        pass

    def test_default_Property(self):
        self.assertEqual(default_Property(), None)

Don' t do this! Django provides enough functionality for managing fixtures with ease(see [1]), you don´t need to write custom fixture handling. Even in the case you need more than Django fixtures provides, there are some thirty packages you probably want to look before even thinking about write your own code (I really like factory_boy [2]

).

 

The default_Property() function in app1 is just set to `return None` for now, so that the above test should work.

However, the test fails.  I get this strange error:

======================================================================
ERROR: test_default_Property (myproj.app1.tests.FunctionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 508, in __call__
    self._post_teardown()
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 522, in _post_teardown
    self._fixture_teardown()
  File "/usr/local/lib/python2.7/dist-packages/django/test/testcases.py", line 847, in _fixture_teardown
    transaction.leave_transaction_management(using=db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 52, in leave_transaction_management
    connection.leave_transaction_management()
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", line 115, in leave_transaction_management
    raise TransactionManagementError("This code isn't under transaction "
TransactionManagementError: This code isn't under transaction management
----------------------------------------------------------------------

Not really sure what´s going with this error, but I think may be related to your fixtures code and it should disappear if you use Django builtin fixtures.

 

I could not find an obvious solution to this, as I do not even have TransactionMiddleware enabled in my settings.

Not really sure, but I think Django runs tests under transactions and loaddata command probably too, so this error might be due to some conflict with your code.

 

The other issue relates to coverage - it does not seem to recognise I have written this test and still flags it as `red` ... Currently I am using `coverage html' - but how I get it work properly?

How are you running coverage? If you are using django-nose you probably should use nose coverage plugin.

Any help getting started overcoming these issues will be helpful, so I can get on with the actual business of writing tests.  (As a side-note, if there is a good existing code base, for a Django-based project, that has test code I could look at, I'd appreciate a link to it.)

Thanks
Derek

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.

HTH

[1] https://docs.djangoproject.com/en/dev/topics/testing/overview/#django.test.TransactionTestCase.fixtures

[2] https://factoryboy.readthedocs.org/en/latest/

[Derek]

Rafael

I appreciate your reply; below I try and explain further the reasoning behind the approach I took.  If you can respond to that, it would help.


Firstly, the issue of test file locations - there are least two places I have found that recommend these be placed in a different directory:

From http://toastdriven.com/blog/2011/apr/17/guide-to-testing-in-django-2/

"I prefer to split up the tests into a file structure that mirrors the app's setup. To do this, we create a new tests directory then move the existing tests.py into tests/views.py."

From https://pycon-2012-notes.readthedocs.org/en/latest/testing_and_django.html

"Don’t make the mistake of putting the tests for your app in a tests.py, because then other people who use your app will be forced to run your tests when they don’t really need to. Better to put your tests in another directory."

So this approach should be OK; and the tests seem work - but with that strange error message.  Still not sure what the exact cause is...


Thanks for the factory_boy suggestion.  I had looked at it, but was trying to avoid having to learn too may things all at once!  The code I have used for loading my fixtures comes from http://stackoverflow.com/questions/979434/how-to-load-fixtures-only-once-in-django-unit-tests  - and I added that because of the exact problem raised in that question; that Django reloads the fixtures for every single test; and that just seems to slow things right down (we have a lot of fixtures we need for our database).  if you can suggest another or better way to load our fixtures just once for a whole class of tests, I'd appreciate hearing about that.

Thanks
Derek

[Rafael Durán Castañeda]

Hi,

El 21/09/2013, a las 13:36, Derek <game...@gmail.com> escribió:

Rafael

I appreciate your reply; below I try and explain further the reasoning behind the approach I took.  If you can respond to that, it would help.


Firstly, the issue of test file locations - there are least two places I have found that recommend these be placed in a different directory:

From http://toastdriven.com/blog/2011/apr/17/guide-to-testing-in-django-2/

"I prefer to split up the tests into a file structure that mirrors the app's setup. To do this, we create a new tests directory then move the existing tests.py into tests/views.py."

It doesn´t matter if app.tests is a module or a package (some test runners will need tests being importable from app.tests in order to work), so that approach still uses default location as I suggested.

From https://pycon-2012-notes.readthedocs.org/en/latest/testing_and_django.html

"Don’t make the mistake of putting the tests for your app in a tests.py, because then other people who use your app will be forced to run your tests when they don’t really need to. Better to put your tests in another directory."

I don´t agree this, since people who don´t want run app.tests can just use a test runner that runs only your tests (as django-nose and pytest-django do); even if you don´t want your tests to be run, you can just exclude them in the setup.py.

So this approach should be OK; and the tests seem work - but with that strange error message.  Still not sure what the exact cause is…

Looking further to stackoverflow example, I´ve got the same error as you, but I managed  to solve it just deleting the _fixture_setup method, so my guess was wrong (since there is no conflict with loading/flushing data this way) but  disabling _fixture_setup seems to break Django tests transaction management (tested with Django 1.5).

Thanks for the factory_boy suggestion.  I had looked at it, but was trying to avoid having to learn too may things all at once!  The code I have used for loading my fixtures comes from http://stackoverflow.com/questions/979434/how-to-load-fixtures-only-once-in-django-unit-tests  - and I added that because of the exact problem raised in that question; that Django reloads the fixtures for every single test; and that just seems to slow things right down (we have a lot of fixtures we need for our database).  if you can suggest another or better way to load our fixtures just once for a whole class of tests, I'd appreciate hearing about that.

If you need load fixtures just once, you can use initial_data fixture (you can change fixture directories setting in your test settings if you don´t want mess with your production data). However if you need different fixtures to be loaded depending on your test module, your approach may be the right choice; thought you probably want to check django-nose fast fixtures https://github.com/jbalogh/django-nose#enabling-fast-fixtures or just create DB objects programmatically (using factory_boy or whatever).

HTH,

Rafael

[Derek]

Hi Rafael

Yes, you are right: deleting the _fixture_setup method allows the tests to run successfully!

I also do have all my fixture data stored in 'appname/fixtures/initial_data.json' so I assume from what you say that this data will only be loaded once for all the tests.

Once I have more confidence with further tests, I will also try out factory_boy for other, non-fixture, data I need to test.

Thanks again for all the help here.

Derek

[Derek]

Hi 

Yes, you are right: deleting the _fixture_setup method allows the tests to run successfully! 

I also do have all my fixture data stored in 'appname/fixtures/initial_data.json' so I assume from what you say that this data will only be loaded once for all the tests.

Once I have more confidence with further tests, I will also try with factoryboy for other, non-fixture, data I need to test.

Thanks again for all the help here.

Derek