I have an app which allows users to rate/vote for any kind of object.
Now, to test this app I need to have a fake model contained in a fake
app. Here's the file structure of my app:
Here is the testing code (located in myapp/tests/__init__.py):
import sys
from django.test import TestCase
from django.conf import settings
from django.core.management import call_command
from django.db.models.loading import load_app
def test_blah(self):
item = FakeItem.objects.create(name="blah")
#Do some testing here...
So, all this works pretty well. The 'fakeapp' app is loaded
dynamically, tables are created and the FakeItem model can be use in
my tests.
But it feels dirty. The dynamically created tables can potentially
clash with some other applications (what if someone calls their app
'fakeapp'?). To make it safer maybe I could give a random name to that
app (e.g. 'uytwe876435gjhgdfs0908').
Is there a cleaner way to do this? In a similar app (django-voting
[1]) there is a dedicated test suite, but I'd like to avoid that if
possible.
On Thu, Nov 27, 2008 at 8:33 AM, Julien Phalip <jpha...@gmail.com> wrote:
> So, all this works pretty well. The 'fakeapp' app is loaded > dynamically, tables are created and the FakeItem model can be use in > my tests.
> But it feels dirty. The dynamically created tables can potentially > clash with some other applications (what if someone calls their app > 'fakeapp'?). To make it safer maybe I could give a random name to that > app (e.g. 'uytwe876435gjhgdfs0908').
> Is there a cleaner way to do this? In a similar app (django-voting > [1]) there is a dedicated test suite, but I'd like to avoid that if > possible.
You aren't the first person to have this need: this is logged as ticket #7835. The problem exists for any project that works at the meta level, working on models rather than with models. For example contrib.admin needs test models in order to validate that the admin can display different types of models correctly.
contrib.admin works around the problem by putting the tests in the Django system test suite, rather than in the contrib app itself. Based on your description, this sounds like the same approach Django-voting has used. This approach works fine if you're willing to maintain an external test suite, but doesn't provide an integrated test suite that can be easily distributed with your app.
This is one of those problems that I want to solve, but I haven't yet given a great deal of though into how to solve. I've got a few vague ideas banging around in my head about allowing test application and test model definitions to be included in the test module for an application, but those ideas aren't fully formed. Anyone that wants to help out with some concrete suggestions (and even better - code) is more than welcome to do so.
> On Thu, Nov 27, 2008 at 8:33 AM, Julien Phalip <jpha...@gmail.com> wrote:
> > So, all this works pretty well. The 'fakeapp' app is loaded
> > dynamically, tables are created and the FakeItem model can be use in
> > my tests.
> > But it feels dirty. The dynamically created tables can potentially
> > clash with some other applications (what if someone calls their app
> > 'fakeapp'?). To make it safer maybe I could give a random name to that
> > app (e.g. 'uytwe876435gjhgdfs0908').
> > Is there a cleaner way to do this? In a similar app (django-voting
> > [1]) there is a dedicated test suite, but I'd like to avoid that if
> > possible.
> You aren't the first person to have this need: this is logged as
> ticket #7835. The problem exists for any project that works at the
> meta level, working on models rather than with models. For example
> contrib.admin needs test models in order to validate that the admin
> can display different types of models correctly.
> contrib.admin works around the problem by putting the tests in the
> Django system test suite, rather than in the contrib app itself. Based
> on your description, this sounds like the same approach Django-voting
> has used. This approach works fine if you're willing to maintain an
> external test suite, but doesn't provide an integrated test suite that
> can be easily distributed with your app.
> This is one of those problems that I want to solve, but I haven't yet
> given a great deal of though into how to solve. I've got a few vague
> ideas banging around in my head about allowing test application and
> test model definitions to be included in the test module for an
> application, but those ideas aren't fully formed. Anyone that wants to
> help out with some concrete suggestions (and even better - code) is
> more than welcome to do so.
> Yours,
> Russ Magee %-)
Thank you Russell for pointing me to this ticket. I'm quite keen to
find a solution for this, so I will give it some thought and maybe
give a shot at writing some code.
I took a similar path. I cloned the run_tests function from
django.tests.simple.py and put it in my project directory as
test_setup.py while adding this near the top of the function:
if settings.TEST_APPS:
settings.INSTALLED_APPS += settings.TEST_APPS
That at least allows me to define TEST_APPS in my main settings file
for the project and in a location near INSTALLED_APPS so that I am
less likely to introduce unintended collisions. But my approach feels
dirty too. It shouldn't take much work to incorporate a TEST_APPS
construct into the existing test framework. I realize it might be
nicer if each batch of unit tests could define their own set of extra
test models, but I think that a master list of additional test apps
like above has the advantage of simplicity. But I haven't given this a
lot of thought beyond this simple minded hack.
I'm keen to try to find some proper ways to fix this, but before I go
too deep into the bowels of Django's testing framework, I was
wondering what would be the criticisms you'd have about the "solution"
I came up with (at the start of this thread)? It sort of works without
patching Django, but is it acceptable? and if not, why?
At this stage, it looks good enough to me except for two main points
that feel dirty:
1) the test models should be unloaded after the test is run
2) potential conflicts with external apps should be avoided.
Any advice/hint welcome ;)
Julien
On Nov 27, 12:42 pm, Rock <r...@rockhoward.com> wrote:
> I took a similar path. I cloned the run_tests function from
> django.tests.simple.py and put it in my project directory as
> test_setup.py while adding this near the top of the function:
> if settings.TEST_APPS:
> settings.INSTALLED_APPS += settings.TEST_APPS
> That at least allows me to define TEST_APPS in my main settings file
> for the project and in a location near INSTALLED_APPS so that I am
> less likely to introduce unintended collisions. But my approach feels
> dirty too. It shouldn't take much work to incorporate a TEST_APPS
> construct into the existing test framework. I realize it might be
> nicer if each batch of unit tests could define their own set of extra
> test models, but I think that a master list of additional test apps
> like above has the advantage of simplicity. But I haven't given this a
> lot of thought beyond this simple minded hack.
On Thu, Nov 27, 2008 at 11:21 AM, Julien Phalip <jpha...@gmail.com> wrote:
> I'm keen to try to find some proper ways to fix this, but before I go > too deep into the bowels of Django's testing framework, I was > wondering what would be the criticisms you'd have about the "solution" > I came up with (at the start of this thread)? It sort of works without > patching Django, but is it acceptable? and if not, why?
> At this stage, it looks good enough to me except for two main points > that feel dirty: > 1) the test models should be unloaded after the test is run > 2) potential conflicts with external apps should be avoided.
I will openly admit that I haven't given this much though, except in the abstract. I'm certain that there are many details and gotchas that I haven't thought of that implementation will reveal.
Broadly speaking, I'm happy with the approach you are suggesting. The interface needs some work, though. I'd like to think that at an API level, it could be as simple as:
class MyMetaTest(TestCase): apps = ['fakeapp','otherapp']
def test_stuff(self): ...
where 'fakeapp' et al are submodules of the test module that contain a models.py definition. Obviously, the test applications need to be: - Added to INSTALLED APPS and the app cache on startup - Installed in the app cache before the syncdb caused by the pre-test database flush takes effect. You shouldn't need to manually invoke syncdb. - Removed from INSTALLED_APPS and the app cache on teardown
I'm of two minds as to whether apps should define a complete replacement for INSTALLED_APPS, or if it should be a supplement to the INSTALLED_APPS that already exists for the project. This hits up against the recurring issue of testing as a per-app process vs testing as a project integration process.
The name clash problem you describe shouldn't be that big an issue - you have the django-admin model validation tools at your disposal. You will probably want to put in some checks to make sure that validation only gets called once (so that you're not needlessly revalidating), but the validator should pick out any name clashes or other conflicts.
> On Thu, Nov 27, 2008 at 11:21 AM, Julien Phalip <jpha...@gmail.com> wrote:
> > I'm keen to try to find some proper ways to fix this, but before I go
> > too deep into the bowels of Django's testing framework, I was
> > wondering what would be the criticisms you'd have about the "solution"
> > I came up with (at the start of this thread)? It sort of works without
> > patching Django, but is it acceptable? and if not, why?
> > At this stage, it looks good enough to me except for two main points
> > that feel dirty:
> > 1) the test models should be unloaded after the test is run
> > 2) potential conflicts with external apps should be avoided.
> I will openly admit that I haven't given this much though, except in
> the abstract. I'm certain that there are many details and gotchas that
> I haven't thought of that implementation will reveal.
> Broadly speaking, I'm happy with the approach you are suggesting. The
> interface needs some work, though. I'd like to think that at an API
> level, it could be as simple as:
> class MyMetaTest(TestCase):
> apps = ['fakeapp','otherapp']
> def test_stuff(self):
> ...
> where 'fakeapp' et al are submodules of the test module that contain a
> models.py definition. Obviously, the test applications need to be:
> - Added to INSTALLED APPS and the app cache on startup
> - Installed in the app cache before the syncdb caused by the pre-test
> database flush takes effect. You shouldn't need to manually invoke
> syncdb.
> - Removed from INSTALLED_APPS and the app cache on teardown
> I'm of two minds as to whether apps should define a complete
> replacement for INSTALLED_APPS, or if it should be a supplement to the
> INSTALLED_APPS that already exists for the project. This hits up
> against the recurring issue of testing as a per-app process vs testing
> as a project integration process.
> The name clash problem you describe shouldn't be that big an issue -
> you have the django-admin model validation tools at your disposal. You
> will probably want to put in some checks to make sure that validation
> only gets called once (so that you're not needlessly revalidating),
> but the validator should pick out any name clashes or other conflicts.
> Russ %-)
Thanks a lot for the feedback and pointers. I'll think about it over
the next few days and will try to post a first sketch patch in the
ticket.
