How to run all tests in INSTALLED_APPS with the new test runner 1.6?

[Kyle MacFarlane]

With the new test runner in 1.6 it only seems to be running tests under my project and ignoring all tests in dependent apps listed in INSTALLED_APPS.

If I try to specify another app to test by doing something like "django test a_dependency" I get an exception from Python's unittest saying "Path must be within the project" (all my dependencies are installed system wide or in a shared buildout folder in my user dir).

The only tests outside of my project I seem to be able to run are Django's itself via "django test django".

How can I get the new test runner to discover and run all tests in INSTALLED_APPS? As far as I can tell the new test runner only supports tests located under the cwd which is not very useful for me since my projects tend to just amalgamate smaller apps located elsewhere.

[Russell Keith-Magee]

Hi Kyle,

The short answer is no, you can't. However, there's method in this madness.

Essentially *all* Django projects are amalgams of smaller apps from elsewhere, so you're not alone in this. However, it doesn't follow that you need to run the tests for those apps. Consider the tests for one of the many apps in your project. This app is either:

1) A standalone, third party app, provided by another developer. This developer has presumably written and run the test suite for that app, validated that they pass, and packaged the app as version X. You pin version X in your requirements file. At this point, there's no value in you running these tests as part of your own project -- these tests aren't going to start spontaneously failing just because they're in your project.

2) A standalone app that you're maintaining in house. Although you're maintaining this in house, it's really no different to case 1. Good discipline means writing code, running tests, and validating a release (either as a tagged version, or just a VCS hash/revision) and pinning at that release. Again, the testing of the app is (or should be) independent of the testing of your project.

3) An app that is strongly tied to this particular project. In which case, it should be part of the project directory, and will be picked up using Django's test suite. If you want to put your project directory somewhere other than the default, you can use the -t option to ./manage.py test to point at your top level project directory. This doesn't change the discovery strategy; it just alters the place that Django looks.

So - you shouldn't have any need to run *all* the tests in INSTALLED_APPS -- just the apps that are actually part of your project, and won't be independently tested. If the app is independently maintained, but not independently tested, you've got much bigger problems :-)

Yours,

Russ Magee %-)

[jordi.gutie...@ecometrica.org]

Hey, sorry to respond to such an old thread, but now that 1.8 completely removed the old test runner, I am wondering if there's any way to get it back.

I don't want to explain why I want it back. Please don't tell me I don't want it.

So, starting from the axiom that I want it back, how should I do it? Grab it from git history?

[Gergely Polonkai]

Hello,

do you mean the integrated unittest module, or the Django test runners?

However, both needs to be fetched from old history, and I daresay it won't be an easy fetch. 1.6 code will more than likely be incompatible with 1.8, so even if you backmerge it, you will have to do a lot of patching.

Best,
Gergely

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/df0a8e3c-129e-4db4-8e8a-12d6d6ce7573%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jordi.gutie...@ecometrica.org]

I want the old behaviour of running all tests in INSTALLED_APPS by default Actually, I don't care too much if this is accomplished by grabbing the old django.test.simple.DjangoTestSuiteRunner from git history or by any other means.

[Gergely Polonkai]

I'm not sure I understand what you want to achieve. I use 1.8 and it runs all tests for all apps without backporting anything.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f8053bf8-0008-40e7-8794-8d536dfaf621%40googlegroups.com.

[Tim Graham]

Copying the old test runner from an old version of Django seems like a fine solution. I thought someone might have published it on PyPI, but I couldn't find it if so.

[jordi.gutie...@ecometrica.org]

No, the new test runner does not run any 3rd party apps not in my project, and it does not run django.contrib either.

On Wednesday, October 7, 2015 at 2:06:08 AM UTC-4, Gergely Polonkai wrote:

[jordi.gutie...@ecometrica.org]

Okay, I'll try grabbing the old test runner from git, thanks.

[Carl Meyer]

On 10/07/2015 06:57 AM, jordi.gutie...@ecometrica.org wrote:
> Okay, I'll try grabbing the old test runner from git, thanks.

That should work fine.

You should be aware, though, that specifically in the case of the
django.contrib app tests, we make no promise that they will pass under
your project's configuration, and pull requests that complexify the
tests solely to make them pass under a wider variety of configurations
will not be accepted. It is our policy that they should pass when run as
part of the Django test suite, but not necessarily in an arbitrary project.

You may find that a similar de facto policy exists for an increasing
number of third-party apps' test suites, too.

Carl

[jordi.gutie...@ecometrica.org]

On Wednesday, October 7, 2015 at 10:16:30 AM UTC-4, Carl Meyer wrote:

You should be aware, though, that specifically in the case of the
django.contrib app tests, we make no promise that they will pass under
your project's configuration,

That's fine. I just want to make sure we didn't break anything.

[Tim Graham]

And actually as of Django 1.8, the contrib tests aren't shipped inside each application directory, so they won't be discovered even if you switch to the old test runner.