Testing documentation

[Daniele Procida]

I have done some work on writing up a tutorial for testing (part 5 of the existing tutorial, in effect).

My draft so far:

<https://github.com/evildmp/django/blob/testing-tutorial/docs/intro/tutorial05.txt>

I'd really appreciate feedback, on any aspect of this.

Trac ticket is <https://code.djangoproject.com/ticket/9962>.

Thanks,

Daniele

[Tim Graham]

This looks great, thanks for your work on this!  I've made some minor revisions and put up an HTML version of the new tutorial for easier review.

http://techytim.com/django/9962/intro/tutorial05.html

[Tim Graham]

Bumping this in hopes of getting a couple more reviews.

Please leave comments on the ticket or pull request:

https://code.djangoproject.com/ticket/9962

https://github.com/django/django/pull/548

[James Pic]

It's great! I just have a few questions:

- would it be useful to mention that if the app lives in a separate (ie. open source) repo, it will need the repo to contain a test_project to run `./manage.py test theapp` in CI systems ?

- would it be useful to add an example .travis.yml and some info about setting up travis-ci (if the app is open source of course)

Keep up the great work !

[Aymeric Augustin]

Hi Tim,

I just reviewed the online version at http://techytim.com/django/9962/intro/tutorial05.html and I found it very clear.

Here are a few suggestions to make it even better. Some don't fit well in inline comments, so I wrote an email instead; I hope you don't mind.

The "Why you need to create tests" section is written from the point of view of a single developer. Complex applications will be maintained by teams. Tests guarantee that colleagues don't inadvertently break your code (and that you don't break theirs without knowing). If you want to make a living as a Django programmer, you must be good at writing tests!

Another argument — but a weaker one, and I know you can't extend that section infinitely — is that tests help identify fragile or even dead code. If you can't test a piece of code, it usually means that code should be refactored or removed. That idea could be postponed to a paragraph introducing coverage.py.

I recommend to remove the "descriptive comments" in code snippets, such as this one:

    # was_published_recently() should return False

    self.assertEqual(future_poll.was_published_recently(), False)

It isn't useful to paraphrase the code in comments. This applies to all test snippets in the tutorial.

A small style suggestion:

    def was_published_recently(self):

        return (self.pub_date >= timezone.now() - datetime.timedelta(days=1)

            and self.pub_date < timezone.now())

The tests cases in "Testing our new view" should take advantage of assertQuerysetEqual to make the assertions more readable.

https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.TestCase.assertQuerysetEqual

It's the first time I encounter the pattern: "initialize objects in setUp, save them in test cases when you need them in the database". It's a bit weird. I think most people would use a factory, which can be as simple as:

    def create_poll(self, question, days):

        Poll.objects.create(question=question, pub_date=timezone.now() + datetime.timedelta(days=days))

The stance in "When testing, more is better" is extreme, but that's probably what beginners need to hear. Concepts such as mocking, testing layers in isolation, unit vs. functional vs. integrations testing, using inheritance to test combinations, etc. don't belong in a tutorial.

Finally, in "Further testing", it would be worth linking to http://nedbatchelder.com/code/coverage/ and explaining that it's a good way to spot untested parts of your application.

Best regards,

-- 

Aymeric.

[Aymeric Augustin]

Le 1 déc. 2012 à 09:19, James Pic <jame...@gmail.com> a écrit :

- would it be useful to mention that if the app lives in a separate (ie. open source) repo, it will need the repo to contain a test_project to run `./manage.py test theapp` in CI systems ?

Yes, it's a good practice to include required files and to document how to run the tests.

In practice a settings file, and often a URLconf, are enough. See https://github.com/aaugustin/django-resto/blob/master/Makefile#L1 for an example. 

The concept of a "Django project" doesn't help developers structure their apps properly and take advantage of pluggable apps and packaging tools, let's avoid it!

- would it be useful to add an example .travis.yml and some info about setting up travis-ci (if the app is open source of course)

Travis builds are very hard to debug, making the first experience daunting even for experienced developers. I wouldn't recommend it to beginners in a tutorial. Besides, Django's docs don't recommend external services until they're well established.

-- 

Aymeric.

[Tim Graham]

Aymeric, thanks for the review and feedback!

The "Why you need to create tests" section is written from the point of view of a single developer. Complex applications will be maintained by teams. Tests guarantee that colleagues don't inadvertently break your code (and that you don't break theirs without knowing). If you want to make a living as a Django programmer, you must be good at writing tests!

 

Done 

Another argument — but a weaker one, and I know you can't extend that section infinitely — is that tests help identify fragile or even dead code. If you can't test a piece of code, it usually means that code should be refactored or removed. That idea could be postponed to a paragraph introducing coverage.py.

 

Done 

I recommend to remove the "descriptive comments" in code snippets, such as this one:

    # was_published_recently() should return False

    self.assertEqual(future_poll.was_published_recently(), False)

It isn't useful to paraphrase the code in comments. This applies to all test snippets in the tutorial.

Converted these to docstrings as appropriate.

 

A small style suggestion:

    def was_published_recently(self):

        return (self.pub_date >= timezone.now() - datetime.timedelta(days=1)

            and self.pub_date < timezone.now())

Done

 

The tests cases in "Testing our new view" should take advantage of assertQuerysetEqual to make the assertions more readable.

https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.TestCase.assertQuerysetEqual

 

Done

 

It's the first time I encounter the pattern: "initialize objects in setUp, save them in test cases when you need them in the database". It's a bit weird. I think most people would use a factory, which can be as simple as:

    def create_poll(self, question, days):

        Poll.objects.create(question=question, pub_date=timezone.now() + datetime.timedelta(days=days))

 

First time I came across this pattern as well.  I've switched to using a factory method.  I didn't make it a method on the class since it's used in multiple classes and didn't want to dive into multiple inheritance.

 

The stance in "When testing, more is better" is extreme, but that's probably what beginners need to hear. Concepts such as mocking, testing layers in isolation, unit vs. functional vs. integrations testing, using inheritance to test combinations, etc. don't belong in a tutorial.

Finally, in "Further testing", it would be worth linking to http://nedbatchelder.com/code/coverage/ and explaining that it's a good way to spot untested parts of your application.

 

Done