Re: [Django] #7835: Provide the ability for model definitions that are only availably during testing
Django
testing
-------------------------------------+-------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New | Status: new
feature | Component: Testing framework
Milestone: | Severity: Normal
Version: SVN | Keywords: feature test models
Resolution: | Has patch: 0
Triage Stage: Accepted | Needs tests: 0
Needs documentation: 0 | Easy pickings: 0
Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
Changes (by kmike):
* cc: kmike84@… (added)
* ui_ux: => 0
* easy: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:30>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
testing
-------------------------------------+------------------------------------
Type: New feature | Status: new
Component: Testing framework | Version: SVN
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by gonz):
* cc: gonz (removed)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:31>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by adrian_lc):
With 1.6 and the new discovery-runner the models in `tests.py` are not
being installed anymore, so this feature would be way more useful now.
Just leaving an idea here, what about a decorator just like
`override_settings` but for installing models?
The usage would be like:
{{{#!python
class SomeTestModel(models.Model):
# whatever
class AnotherTestModel(models.Model):
# whatever
class YetAnotherTestModel(models.Model):
# whatever
@test_models(SomeTestModel)
class SomeModelsTestCase(TestCase):
@test_models(AnotherTestModel)
def test_them(self):
with test_models(YetAnotherTestModel):
# The 3 models available here
}}}
Of course this wouldn't actually install the app, just models for testing
custom fields or abstract models.
I actually have some code done but it's far from being correctly tested
and I'm not a django developer (actually a newbie user), so my solution is
sure to be terrible.
Anyway, anticipating some issues I encountered:
* Main problem was the automatic rollback from `TestCase`. My database
(postgres) rejects transactions with changes to a table followed by a
table drop.
* When applied to a test method (or as a context manager), I couldn't find
a way to detect if the test function comes from a `TransactionTestCase` or
a `TestCase`.
* Finally managed something with `restore_transaction_methods` and
`disable_transaction_methods` from `django.test.testcases`.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:32>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: adrianlopezcalvo@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:33>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by JocelynD):
Replying to [comment:32 adrian_lc]:
> With 1.6 and the new discovery-runner the models in `tests.py` are not
being installed anymore, so this feature would be way more useful now.
I don't get it, I did a minimal test case (attached) with django 1.6:
* a "buggy" project
* a "tested" app
* tested/models.py contains a single abstract model `AbstractTestStat`
* tested/tests.py contains a `TestStat` model inheriting from
`AbstractTestStat`
* tested/tests.py contains a single test testing TestStat
Either
./manage.py test
or
./manage.py test tested
Works fine, populating the test db and succeeding test as expected.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:34>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by JocelynD):
However, I don't know why but when using south, the "Test Models in
tests.py" trick does not work (table is not created), so you have to put
in your ''settings.py'' the following :
{{{
SOUTH_TESTS_MIGRATE = False
}}}
to disable the south migrations for the test db.
(which will anyway save you some cpu time...)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:35>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by glassglaze@…):
Replying to [comment:34 JocelynD]:
>
> I don't get it, I did a minimal test case (attached) with django 1.6:
>
> * a "buggy" project
> * a "tested" app
> * tested/models.py contains a single abstract model `AbstractTestStat`
> * tested/tests.py contains a `TestStat` model inheriting from
`AbstractTestStat`
> * tested/tests.py contains a single test testing TestStat
>
> Works fine, populating the test db and succeeding test as expected.
>
It seems like you don't interact with db (save models to test db). I
didn't manage to run tests. Do you think I've missed something.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:36>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by timo):
I've marked #22459 "Creating model classes for test purposes breaks
migrations" as a duplicate of this.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:37>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by mitchell):
I managed to work around this using migrations instead of syncdb under
Django 1.7 with the following:
{{{
from django.conf import settings
from django.db import migrations
IS_TEST_DB = settings.DATABASES.get(
'default', {}).get('NAME', '').startswith('test_')
class Migration(migrations.Migration):
dependencies = [...]
operations = [...]
if IS_TEST_DB:
operations.extend([
...
])
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:38>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: daniel.samuels1@… (added)
Comment:
This functionality is still broken as of Django 1.7.7. This is a very
basic repro example:
{{{#!python
from django.db import models
from django.test import TestCase
class TestModel(models.Model):
label = models.CharField(
max_length=100,
)
class Test(TestCase):
def setUp(self):
TestModel.objects.create(
label='X'
)
}}}
Add this to any app as tests.py and run it, you will get an error like
this:
{{{
django.db.utils.ProgrammingError: relation "media_testmodel" does not
exist
LINE 1: ...a_testmodel"."id", "media_testmodel"."label" FROM "media_tes...
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:39>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by timgraham):
A [https://github.com/django/django/pull/4630/ documentation patch] has
been proposed based on comment 38's approach of conditionally adding
migration operations based on the database name. I don't think this
approach is something we should officially recommend. I'd rather wait for
a proper solution. One idea is a `@test_model` model class decorator that
the migrations system respects.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:40>
Django
testing
-------------------------------------+------------------------------------
Reporter: russellm | Owner: kkubasik
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: maxime.lorant@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:41>
Django
testing
-------------------------------------+------------------------------------
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by Ashley Waite):
This was a particularly annoying issue to me, which I resolved by creating
a new TestRunner that did the work for me.
I think including something like this for easy use would effectively
resolve this.
{{{
from importlib.util import find_spec
import unittest
from django.apps import apps
from django.conf import settings
from django.test.runner import DiscoverRunner
class TestLoader(unittest.TestLoader):
""" Loader that reports all successful loads to a runner """
def __init__(self, *args, runner, **kwargs):
self.runner = runner
super().__init__(*args, **kwargs)
def loadTestsFromModule(self, module, pattern=None):
suite = super().loadTestsFromModule(module, pattern)
if suite.countTestCases():
self.runner.register_test_module(module)
return suite
class RunnerWithTestModels(DiscoverRunner):
""" Test Runner that will add any test packages with a 'models' module
to INSTALLED_APPS.
Allows test only models to be defined within any package that
contains tests.
All test models should be set with app_label = 'tests'
"""
def __init__(self, *args, **kwargs):
self.test_packages = set()
self.test_loader = TestLoader(runner=self)
super().__init__(*args, **kwargs)
def register_test_module(self, module):
self.test_packages.add(module.__package__)
def setup_databases(self, **kwargs):
# Look for test models
test_apps = set()
for package in self.test_packages:
if find_spec('.models', package):
test_apps.add(package)
# Add test apps with models to INSTALLED_APPS that aren't already
there
new_installed = settings.INSTALLED_APPS + tuple(ta for ta in
test_apps if ta not in settings.INSTALLED_APPS)
apps.set_installed_apps(new_installed)
return super().setup_databases(**kwargs)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:42>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Ashley Waite (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:43>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Carlos Palol (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:44>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: direx (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:45>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Simon Charette (added)
Comment:
While working on a workaround for this I came up with a non-invasive
solution that might be acceptable to resolving the ticket.
The idea is similar to Ashley's solution but is more explicit as it
require a function call in the `app/tests/__init__.py` module. It does
however isolate each app into their own app, which prevent name
collisions, and doesn't require the `app_label = 'test'` assignment on
each test model.
The solutions boils down to this function
{{{#!python
def setup_test_app(package, label=None):
"""
Setup a Django test app for the provided package to allow test models
tables to be created if the containing app has migrations.
This function should be called from app.tests __init__ module and pass
along __package__.
"""
app_config = AppConfig.create(package)
app_config.apps = apps
if label is None:
containing_app_config = apps.get_containing_app_config(package)
label = f'{containing_app_config.label}_tests'
if label in apps.app_configs:
raise ValueError(f"There's already an app registered with the
'{label}' label.')
app_config.label = label
apps.app_configs[app_config.label] = app_config
app_config.import_models()
apps.clear_cache()
}}}
Which when called from `app/tests/__init__.py` as
`setup_test_app(__package__)` will create an `app_tests` appconfig entry
and auto-discover the models automatically. Since the `*.tests` modules
should only be loaded on test discovery the app and its models will only
be available during tests. Keep in mind that if your test models reference
models from an application with migrations you'll also need to manually
create migrations for these tests models but once that's done you should
be good to go.
It does feel less magic and convention based than Ashley's solution as it
prevents conflicts between models and allows multiple test apps per app
from any test package structure. Thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:46>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Mark Gregson (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:47>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by Ahmad Abdallah):
Your solution is indeed less magic and much more isolated; reusing
automatic model discovery from appConfig makes a lot more sense than
attempting load them manually.
This will be a nice feature to have for 3.2.
The manual secondary migrations for models referenced outside of the test
app seems like a fair cost to get models inside tests, and from my
understanding, the test models are meant to be self-contained i.e models
created purely for tests that will reference each other only so it
shouldn't be much of an issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:48>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by Filipe Pina):
Thank you Simon for that clean life saving piece of code!
Been using for a few months and just today had to use a test child model
of a real model so bumped into the lack of migrations.
Just defined it in the main models, ran makemigrations and moved that new
migration into “tests/migrations”, worked perfectly.
Hope this makes it into a release “soon” :)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:49>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Carlton Gibson (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:50>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: Keryn Knight (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:51>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
* cc: HarryKane (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:52>
Django
testing
-------------------------------------+------------------------------------
Reporter: Russell Keith-Magee | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Comment (by HarryKane):
Replying to [comment:46 Simon Charette]:
> Keep in mind that if your test models reference models from an
application with migrations you'll also need to manually create migrations
for these tests models but once that's done you should be good to go.
Could you elaborate on how to manually create the migration for the test
model?
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:53>
Django
testing
Reporter: Russell Keith-Magee | Owner: Josh
| Thomas
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Josh Thomas):
* owner: nobody => Josh Thomas
* status: new => assigned
Comment:
Hard to believe this ticket hasn't been picked up yet. I've been using
Simon's provided workaround for at least two years, maybe longer.
Any idea what would be needed for this to make it in to Django proper?
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:54>
Django
testing
-------------------------------------+-------------------------------------
Reporter: Russell Keith-Magee | Owner: Josh
| Thomas
Type: New feature | Status: assigned
Component: Testing framework | Version: dev
Severity: Normal | Resolution:
Keywords: feature test models | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
@HarryKane, sorry I missed your comment
> Could you elaborate on how to manually create the migration for the test
model?
I had to manually create the migration files myself. Since the models are
only auto-discovered during tests it's kind of painful to ''trick'' the
migration system in creating the migration itself; you need to perform
test auto-discovery before running `makemigrations`. Maybe it can be a
limitation of the solution at first?
@Josh Thomas
> Any idea what would be needed for this to make it in to Django proper?
Since no other solution emerged for that long I'd suggest creating a PR
with the above solution available from `django.test` and documenting its
usage. I've used it with success in many projects and I've been wanting to
create third-party package that exposes it as well as a
`createtestmigrations` command that basically performs test discovery and
then delegates to `makemigrations` but I haven't been able to do so.
The challenge will likely be in writing tests for it. I guess we could
models defined in a specialzed test app that are discovered by the Django
test suite itself and that would be enough?
--
Ticket URL: <https://code.djangoproject.com/ticket/7835#comment:55>