Django TestCase isolation: setUp vs setUpClass

[Danilo Bargen]

Hi all

Today I ran across an issue while debugging Django tests. As stated in the docs, all model changes in a TestCase that are done during the test are reverted between the tests. This creates a nice and useful isolation of the different tests. Each time a test runs, the database looks just the way it was after the syncdb.

> from django.test import TestCase
> from apps.front import models
> 
> class Test1(TestCase):
>     def setUp(self):
>         models.User.objects.create(username='spamham')
> 
>     def test(self):
>         pass
> 
> class Test2(TestCase):
>     def setUp(self):
>         models.User.objects.create(username='spamham')
> 
>     def test(self):
>         pass

These test run fine, even though two users with the same PK are created, because Django handles test isolation. But things behave differently when using the setUpClass classmethod. I used setUpClass instead of setUp because there are certain models that I just want to create once for all the tests in a test case. It would be an unnecessary performance loss if those model instances were created and then rolled back for each test method. And less DRY.

The problem is, model changes that are done in setUpClass aren't rolled back between the test cases.

> from django.test import TestCase

> from apps.front import models

> 

> class Test1(TestCase):

>     @classmethod

>     def setUpClass(cls):

>         models.User.objects.create(username='spamham')

> 

>     def test(self):

>         pass

> 

> class Test2(TestCase):

>     @classmethod

>     def setUpClass(cls):

>         models.User.objects.create(username='spamham')

> 

>     def test(self):

>         pass

This fails with an IntegrityError due to a violation of the front_user_username_key unique constraint, because the User object from the first test wasn't removed.

Is this a design decision? Or a technical limitation? (I realize that the use of a setup classmethod could cause issues with parallelization, but those issues could be addressed in the test runner.)

And in the meantime, is there a better solution than to put all the common setup code in the setUp method?

Note that I don't want to use fixtures for this. Fixtures are not a good way to create test data, as they're very static and need to be updated with the code. This also makes them error-prone. Instead I'm using Model Mommy (https://github.com/vandersonmota/model_mommy), a model factory library for Django. Another good alternative would be Factory Boy. These factories are also the reasons why I don't want to do manual cleanup in tearDownClass: First of all the model factory also creates related models that are needed to satisfy some foreign key constraints. I don't have references to all those model instances. And the second reason is that I think the database cleanup is something that Django should handle, not me.

Danilo

[Russell Keith-Magee]

Neither really. It's an artefact of the history of Django's test framework.

The test framework was written almost 7 years ago, predating unittest2 by several years. setUpClass was introduced by unittest2. As a result, the infrastructure that resets tests after each setUp/tearDown pair hasn't been modified to include setUpClass/tearDownClass as well.

I haven't looked at the problem in detail to establish if there is a fundamental technical limitation preventing this; but off the top of my head, I can think of a few complications that might pose difficulty. In particular, the transaction rollback approach used to speed up fixture loading won't adapt well to having two different 'setup' routines.

 

And in the meantime, is there a better solution than to put all the common setup code in the setUp method?

Unfortuntately not. 

 

Note that I don't want to use fixtures for this. Fixtures are not a good way to create test data, as they're very static and need to be updated with the code. This also makes them error-prone. Instead I'm using Model Mommy (https://github.com/vandersonmota/model_mommy), a model factory library for Django. Another good alternative would be Factory Boy. These factories are also the reasons why I don't want to do manual cleanup in tearDownClass: First of all the model factory also creates related models that are needed to satisfy some foreign key constraints. I don't have references to all those model instances. And the second reason is that I think the database cleanup is something that Django should handle, not me.

 

If you want to avoid fixtures, Factory Boy would be my suggestion; I haven't used Model Mommy myself, but it looks to be in much the same vein.

Yours,

Russ Magee %-)

[Samantha Zeitlin]

I'm having this same issue and was wondering if anyone ever came up with a good solution? Specifically, I'm wondering about using a tearDownClass() method after using Factory Boy objects in my setUpClass() methods? 

[Aymeric Augustin]

Hello Samantha,

As of Django 1.8 there is a built-in solution to this problem:

https://docs.djangoproject.com/en/1.8/topics/testing/tools/#django.test.TestCase.setUpTestData

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1215c89a-2881-401a-97ec-06c92411aa65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.