discontinue shipping tests with contrib apps?

[Tim Graham]

I wonder if there is any need to continue to ship tests as part of contrib apps? Currently the contributing docs say, "Tests for contrib apps go in their respective directories under django/contrib, in a tests.py file. You can split the tests over multiple modules by using a tests directory in the normal Python way." but we are currently inconsistent (see below). Shipping the tests in contrib means that end users may try to run them with their own projects and this often leads to failures if their project's settings do not match the settings that runtests.py expects. (e.g. #24299 asks for us to skip contrib tests that require an 'other' database connection if a project doesn't have that).

apps with tests in tests/
admin
admin_docs
postgres
staticfiles
syndication

apps with tests in contrib/
auth
flatpages
gis
humanize
messages
redirects
sessions
sitemaps
sites

apps with tests in both locations
contenttypes

[Markus Holtermann]

I'm +1 on moving the contrib tests to tests/<app_label>_tests/ .

The respective ticket on Trac is https://code.djangoproject.com/ticket/24293

/Markus

[Preston Timmons]

I think the "need" is mainly conceptual--whether tests are more appropriately grouped with their app or with the other tests. With the discover runner it's uncommon that contrib tests are included in any local test runs.

I do prefer moving all tests into the tests directory. The logic to get test_modules in runtests.py would be simplified quite a bit from it.

[Marc Tamlyn]

+1 to removing tests from contrib itself, so long as they remain obviously separated in the test section in case of future removals from core.

On 9 February 2015 at 15:44, Preston Timmons <preston...@gmail.com> wrote:

I think the "need" is mainly conceptual--whether tests are more appropriately grouped with their app or with the other tests. With the discover runner it's uncommon that contrib tests are included in any local test runs.

I do prefer moving all tests into the tests directory. The logic to get test_modules in runtests.py would be simplified quite a bit from it.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ba79f1b9-678e-4fdf-8f7a-26319e5ac4d3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Curtis Maloney]

Just to shine a light on another perspective- I frequently tell people to look at contrib for "best practices", and including tests within a 3rd party app would fall under that.

--
Curtis

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1EJWzd0G0noSAHFL3ZsOP9kc6He6%2Bb1a5%3DAe3KhxBBfVg%40mail.gmail.com.

[Tim Graham]

Is it a best practice? In my unscientific sampling, none of the following ship tests inside the application directory, but rather in a separate "tests" directory. Or did I misunderstand what you meant?

- Django REST framework
- django-nap
- the apps that split out from contrib (formtools, comments, localflavor)

[Curtis Maloney]

That'll teach me to respond to email before coffee :)

--

C

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4588baf5-2c6f-463c-a994-0ed4fe9a83ba%40googlegroups.com.

[Jannis Leidel]

> On 10 Feb 2015, at 00:12, Tim Graham <timog...@gmail.com> wrote:
>
> Is it a best practice? In my unscientific sampling, none of the following ship tests inside the application directory, but rather in a separate "tests" directory. Or did I misunderstand what you meant?
>
> - Django REST framework
> - django-nap
> - the apps that split out from contrib (formtools, comments, localflavor)

I think there's been a tendency in the past years ever since we saw an increasing adoption of the unittest2 discover runner anecdotally (and maybe also the Nose and pytest runners) to localize reusable app tests outside their main package to not accidentally trigger them when installed in a Django project that uses the old style test runner.

That's also been the reason why I moved the tests outside of formtools, localflavor and almost all my apps – providing tests in app repos makes sense, but to me the reason to put them into the main package is mostly historical, so nothing I would want to continue to promote for the freshly extracted localflavor and formtools apps.

IIRC Carl had a more elaborate explanation at the time we were adding the discover runner to Django, but that's my gist of it.

+0 for moving app tests outside the contrib apps folders but +1 on updating the app docs to promote tests outside the main packages.

Jannis

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4588baf5-2c6f-463c-a994-0ed4fe9a83ba%40googlegroups.com.

[Preston Timmons]

The trickiest tests to move will be the gis tests.

When gis is enabled, django/contrib/gis/tests is added to the discovery path. This doesn't affect which tests are discovered per se--all gis tests are discovered anyway by discovery on the parent app django.contrib.gis--but it causes the later side-effect of django.contrib.gis.tests.geo3d, django.contrib.gis.tests.geoapp, etc. being added to INSTALLED_APPS during the test run. The purpose of this logic is to avoid running migrations when gis isn't enabled.

If the test apps stay as subdirectories in the gis test folder, runtests.py will need to be updated accordingly. If the namespace is made flat instead, we'd need to update runtests.py to do something like filter out gis_* apps from INSTALLED_APPS when not available.

[Carl Meyer]

On 02/10/2015 12:08 AM, Jannis Leidel wrote:
>> On 10 Feb 2015, at 00:12, Tim Graham <timog...@gmail.com> wrote:
>>
>> Is it a best practice? In my unscientific sampling, none of the
>> following ship tests inside the application directory, but rather
>> in a separate "tests" directory. Or did I misunderstand what you
>> meant?
>>
>> - Django REST framework - django-nap - the apps that split out from
>> contrib (formtools, comments, localflavor)
>
> I think there's been a tendency in the past years ever since we saw
> an increasing adoption of the unittest2 discover runner anecdotally
> (and maybe also the Nose and pytest runners) to localize reusable app
> tests outside their main package to not accidentally trigger them
> when installed in a Django project that uses the old style test
> runner.
>
> That's also been the reason why I moved the tests outside of
> formtools, localflavor and almost all my apps – providing tests in
> app repos makes sense, but to me the reason to put them into the main
> package is mostly historical, so nothing I would want to continue to
> promote for the freshly extracted localflavor and formtools apps.
>
> IIRC Carl had a more elaborate explanation at the time we were adding
> the discover runner to Django, but that's my gist of it.

Elaborate explanation? Sounds like me! :-)

The reason I've generally moved tests outside of reusable apps is the
one Jannis mentions - avoiding having the old pre-discovery runner run
them in people's projects. I think this reason becomes less relevant
with time.

Though I generally prefer separating test code from production code
anyway, even without any Django-specific considerations.

Carl

[Tim Graham]

PR is ready for review: https://github.com/django/django/pull/4106

[Shai Berger]

On Monday 09 February 2015 17:38:27 Markus Holtermann wrote:
> I'm +1 on moving the contrib tests to tests/<app_label>_tests/ .
>

I'm +1 on preventing tests from being packaged with the contrib apps;
I'm also +1 on moving them out of the app folders in the source -- +0 for the
move's inherent value, but +1 because it makes the above packaging easier.

But I'm not sure tests/<app_label>_tests/ is the best destination -- I think
that, just like the contrib apps themselves are collected in a folder, their
tests should be too. Either django/contrib/tests/ or tests/contrib/ seem
appropriate.

The point is to keep it clear that the core does not depend on contrib, and to
make it easier to remove all of contrib to prove it.

Shai.

[Tim Graham]

I don't have a strong opinion on a nested structure for the tests; I merely followed the existing convention for contrib tests that were already in tests/. Note that if we do nest, we still need the "_tests" suffix (or some distinction so to prevent duplicate app_labels from the contrib apps themselves). A nested structure will require additional changes to runtests.py to make it "smarter", and of course moving more tests.

[Aymeric Augustin]

Hello,

I find the flat structure in tests/ quite convenient, especially for making large changes, also in combination with shell globbing. Merging modeltests and regressiontests was a win.

I'm not convinced a contrib folder would help much. Reserving foo_* for contrib app foo sounds sufficient.

-- 

Aymeric.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/af3c2b54-8b58-4749-b44e-b4f8d8b4f8b0%40googlegroups.com.