name of test database

[Michael Radziej]

Hi,

I'm checking out the changes from the last three weeks in svn ...
seems like the test suite is a little bit different now ;-)

I can't say much about it now other than I appreciate all the
hard work by Russell, and that there's a nice test framework in
place now.

Only ... I find the naming of the test database a bit
inconvenient. It's created as 'test_' + settings.DATABASE_NAME
(unless you use sqlite).

My problems with this is that it does not allow me to choose a
name on my own. Now I cannot use the old 'django_test_db', since
it doesn't start with 'test_'. And using the default
'test_myprojectdatabase' is not an option at all since that is
used for a different purpose. So I need to contact my admins to
get a new database, set up a settings file dedicated for running
the django test suite, just to do that.

I can work around this, sure, but I'd propose to allow to
overwrite the default test database name for the test suite
somehow, preferably on the command line of runtests.py.

This probably also applies to using the test framework for your
own application (as opposed to django's own test suite), but I
haven't digged into this, yet.

Cheers,

Michael

[Russell Keith-Magee]

On 9/1/06, Michael Radziej <m...@noris.de> wrote:

I can work around this, sure, but I'd propose to allow to
overwrite the default test database name for the test suite
somehow, preferably on the command line of runtests.py.

Thanks for the feedback.

These are reasonable concerns. I have just committed (r3706) a TEST_DATABASE_NAME setting; by default, this has a value of None, which will cause the test database to use the same 'test_' + DATABASE_NAME scheme; however, if you provide a value, that name will be used instead.

This setting is then used by runtests; settings.TEST_DATABASE_NAME is hardcoded to 'django_test_db', just as it was with the old runtests.

Yours,
Russ Magee

[Adrian Holovaty]

On 9/1/06, Russell Keith-Magee <freakb...@gmail.com> wrote:
> These are reasonable concerns. I have just committed (r3706) a
> TEST_DATABASE_NAME setting; by default, this has a value of None, which will
> cause the test database to use the same 'test_' + DATABASE_NAME scheme;
> however, if you provide a value, that name will be used instead.
>
> This setting is then used by runtests; settings.TEST_DATABASE_NAME is
> hardcoded to 'django_test_db', just as it was with the old runtests.

Hey Russ,

Any particular reason you used a setting instead of a simple
command-line parameter? In my view, we should only be adding settings
if absolutely necessary.

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com

[Russell Keith-Magee]

On 9/1/06, Adrian Holovaty <holo...@gmail.com > wrote:

Any particular reason you used a setting instead of a simple
command-line parameter? In my view, we should only be adding settings
if absolutely necessary.

I agree that a command line argument would probably be viable if the problem was restricted to just runtests.py, but it struck me as an example of a bigger problem.

In Michael's case, it sounds like test_myapplication would never be a viable database name (due to some name clash). Given that he knows this, and it will be a persistent condition, asking him to type './manage.py --test_db_name="no_clash_db_name" test' every time he runs his test suite seems a little onerous. He isn't expected to do the same for DATABASE_NAME - a setting lets him configure his preferred name permanently on a per-app basis.

I suspect most users will never need the setting - the default naming policy should be enough. However, the same could be said for other settings, too (e.g., EMAIL_SUBJECT_PREFIX)

As a bonus, the TEST_DATABASE_NAME setting provides a clean mechanism to avoid running the Django system tests in an application's test database.

Does that sound reasonable?

On a related, cross thread note: any comment on the test system signaling benchmarks and the solutions proposed? 

Yours,
Russ Magee %-)

[Adrian Holovaty]

On 9/1/06, Russell Keith-Magee <freakb...@gmail.com> wrote:

> In Michael's case, it sounds like test_myapplication would never be a viable
> database name (due to some name clash). Given that he knows this, and it
> will be a persistent condition, asking him to type './manage.py
> --test_db_name="no_clash_db_name" test' every time he runs
> his test suite seems a little onerous. He isn't expected to do the same for
> DATABASE_NAME - a setting lets him configure his preferred name permanently
> on a per-app basis.
>
> I suspect most users will never need the setting - the default naming policy
> should be enough. However, the same could be said for other settings, too
> (e.g., EMAIL_SUBJECT_PREFIX)
>
> As a bonus, the TEST_DATABASE_NAME setting provides a clean mechanism to
> avoid running the Django system tests in an application's test database.
>
> Does that sound reasonable?

OK, cool -- sounds reasonable. I just wanted to make sure it was
essential / thought-through.

[Ned Batchelder]

Partly on this topic:  I would very much like to use sqlite in-memory databases for testing, even though I use MySQL for deployment.  The speed difference is 10x.  One way to do this is to have a TEST_DATABASE_ENGINE setting, and add logic to create_test_db().

Thoughts?

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com

[Russell Keith-Magee]

On 9/4/06, Ned Batchelder <n...@nedbatchelder.com> wrote:

Partly on this topic:  I would very much like to use sqlite in-memory databases for testing, even though I use MySQL for deployment.  The speed difference is 10x.  One way to do this is to have a TEST_DATABASE_ENGINE setting, and add logic to create_test_db().

Thoughts?

If there were a TEST_DATABASE_ENGINE setting, it would imply that there were analogs of all the other DATABASE_ settings, too. I can't say I'm crazy about this idea. 'Deploy under MySQL, but test under PostgreSQL' isn't a particularly compelling use case for me. IMHO, testing for one database but deploying onto another sort of defeats the purpose of testing.

However, when you are developing unit tests, having a fast turnaround on test execution is certainly desirable; using SQLite in-memory is certainly one way to get this fast turnaround.

Rather than having TEST_DATABASE_ENGINE (which implies that ANY database engine could be used), I would be inclined to have a TEST_IN_MEMORY setting, which forces all tests to be run in SQLite, using an in memory database.

Furthermore, I would be inclined to make this a command line option, rather than a permanent setting. IMHO, tests should always be executed against the _actual_ database that will be used during deployment. Although there _shouldn't_ be any differences between databases, there occasionally will be - either due to kinks in Django, or because you are deploying custom SQL. Providing a command line option would provide a workaround to get fast development turnarounds without compromising the default (read - preferable) testing conditions.

Opinions?

Yours,
Russ Magee %-)

[Ned Batchelder]

I'm not sure there's much value in Django having a "preferred" way for developers to run their tests.  The most valuable tests are the ones that are run day-to-day, and the ORM does a good job isolating the Python code from the details of the database, so allowing fast tests with excellent verisimilitude to the real environment sounds like a good thing to me.  That said, I understand the desire to cut down on too many infrequently-used settings. 

A TEST_IN_MEMORY setting is a good compromise, though I would prefer it to be settable in the settings file, to reduce the need for extra scripts to get all members of the team to perform tests the best way.

--Ned.

[Matthew Flanagan]

I don't see the need for Django specific TEST_* settings when it is
simple enough to do this in myapp/testsettings.py:

from myapp.settings import *
DATABASE_ENGINE='sqlite3'
DATABASE_NAME=':memory:'

and use ./manage.py --settings=myapp.testsettings.

matthew

[Ned Batchelder]

An excellent suggestion!

[Michael Radziej]

Russell Keith-Magee wrote:
> On 9/4/06, Ned Batchelder <n...@nedbatchelder.com> wrote:
>> Partly on this topic: I would very much like to use sqlite in-memory
>> databases for testing, even though I use MySQL for deployment. The speed
>> difference is 10x. One way to do this is to have a TEST_DATABASE_ENGINE
>> setting, and add logic to create_test_db().
>>
>> Thoughts?
>>
>
> If there were a TEST_DATABASE_ENGINE setting, it would imply that there were
> analogs of all the other DATABASE_ settings, too. I can't say I'm crazy
> about this idea. 'Deploy under MySQL, but test under PostgreSQL' isn't a
> particularly compelling use case for me. IMHO, testing for one database but
> deploying onto another sort of defeats the purpose of testing.
>
> However, when you are developing unit tests, having a fast turnaround on
> test execution is certainly desirable; using SQLite in-memory is certainly
> one way to get this fast turnaround.
>
> Rather than having TEST_DATABASE_ENGINE (which implies that ANY database
> engine could be used), I would be inclined to have a TEST_IN_MEMORY setting,
> which forces all tests to be run in SQLite, using an in memory database.

Just my 2c:

In the beginning, I tried to use SQLite for my tests. I soon
found that this would not work without major changes because of
the various nasty differences between the SQL engines. Also, I
need to be sure that everything works under MySQL, so I now even
run the django test suite under MySQL.

All in all, I don't find this option useful for me, though
other's mileage may vary.

Michael

[Russell Keith-Magee]

On 9/4/06, Matthew Flanagan <mattim...@gmail.com> wrote:

and use ./manage.py --settings=myapp.testsettings.

I briefly considered this approach when the TEST_DATABASE_NAME suggestion came up. However, I was concered about the possibility for accidentally nuking your production database. If you ever forget to pass your test settings file to ./manage.py test, your production database will be deleted and rewritten.

However, now that TEST_DATABASE_NAME is in place (providing protection for the production database), this approach seems like a reasonable solution to Ned's problem (i.e., setting up some overrides to allow rapid testing turnaround). It also means that you can optimize any other setting you want without introducing an explosion of command line arguments to manage.py.

Thanks for the suggestion.

Yours,

Russ Magee %-)