Testing parameters in database settings

[Shai Berger]

Hi all,

Django's database settings currently support eleven separate parameters for
testing, all named 'TEST_*', most of them more-or-less backend-specific (in
fact, six -- already a majority -- are Oracle-specific). We have now
discovered[1] that we need even more Oracle-test-specific parameters, for
better control of the creation of the test tablespaces; I suppose there could
be parallels for those in the other backends (for creation of test databases,
the similar-but-not-equivalent feature).

In my opinion, this is beginning to smell; I'd like to collect all these
parameters into their own sub-dictionary, perhaps named "TESTING"; this is, of
course, not backwards-compatible, and I'd like to hear your opinions. An
alternative is to do something just for Oracle, but that would be a little odd
IMO.

The settings that may be affected are:

Non-backend-specific:

TEST_DEPENDENCIES
TEST_MIRROR

Different semantics on different backends:

TEST_NAME[2]

MySQL:

TEST_CHARSET[3]
TEST_COLLATION

PostgreSQL:

TEST_CHARSET[3]

Oracle:

TEST_USER_CREATE
TEST_USER
TEST_PASSWD
TEST_CREATE
TEST_TBLSPACE
TEST_TBLSPACE_TMP

For the curious-but-lazy, the Oracle settings are needed because Oracle does
not do "databases" like other backends do, and so the testing framework needs
a test user and a test tablespace; the TEST_{USER_,}CREATE settigns are
booleans that control if these will be created for the test or just used. The
parameters I want to add now, following #21775[1], would specify the exact
location for the test tablespace files, and their initial and maximum sizes.

Opinions?

Thanks,
Shai.

[1] https://code.djangoproject.com/ticket/21775

[2] On SQLite, the defualt is to use in-memory database (":memory:"), on
others ('test_' + name); on Oracle the setting is used only for reporting to
the user, and has no effect on the database used.

[3] The name TEST_CHARSET is common to MySQL and PostgreSQL, but the
appropriate values are backend-specific.

[Aymeric Augustin]

This has bugged be for some time too. Fix it!

Even if it's just for tests it would be nice to implement a simple deprecation path.

I'd simply call the new key TEST, but whoever writes the patch can choose the name :-)

-- 

Aymeric.

2014/1/14 Shai Berger <sh...@platonix.com>

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/201401141904.43630.shai%40platonix.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Aymeric.

[Marc Tamlyn]

+1, what Aymeric said.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANE-7mVYNJJiYJBdRAy17rgJV0hRh31fkd%3DMzF30kJOgoU3MMg%40mail.gmail.com.

[Michael Manfre]

I'm -0, on sweeping this explosion of settings (mostly for Oracle) under a "TEST" rug, instead of addressing the underlying problem. The entire TEST_* collection of settings is, in my opinion, a broken design that is used to shim a second "test" database configuration in a spot designed for a single database configuration.

Why are we not encouraging people to define different aliases for testing? Many of those TEST_ settings could be given meaning to the database configuration without being specific to running tests. Several of those TEST_ settings would no longer be necessary and several more would be properly rooted under OPTIONS because they are specific to a database's connection.

The only real benefit I see from the current implementation is that TEST_NAME defaults to a prefixed name that should help reduce the chance of mangling a production database. I'd rather see a top level TEST_DATABASES or a IS_TEST_DATABASE (default False) key added to maintain that small amount of configuration safety.

Regards,

Michael Manfre

[Shai Berger]

Hi,

On Tuesday 14 January 2014 21:35:04 Michael Manfre wrote:

> Why are we not encouraging people to define different aliases for testing?
> Many of those TEST_ settings could be given meaning to the database
> configuration without being specific to running tests. Several of those
> TEST_ settings would no longer be necessary and several more would be
> properly rooted under OPTIONS because they are specific to a database's
> connection.
>

The way testing currently works is by creating a throw-away database, running
the tests on that, and throwing it away. This means, among other things, that
you need settings for a database service and credentials for a user that can
create databases on this service. The common use-case is that this user is
your "main" database user, and the service is your main service. Your
suggestion, if I understand it correctly, gives the user two options:

1) Use separate credentials, and perhaps even a separate service, for testing;
this implies that the test database stays alive all the time.
2) Write configurations with repetitions or some other awkwardness, e.g.

default_db = {
'ENGINE': 'this',
'HOST': 'that',
'NAME': 'the_other',
'USER': 'me',
'PASSWORD': 'my secret',
}

DATABASES = {
'default' : default_db,
'test_default' : dict(default_db,
NAME='test_the_other',
IS_TEST=True),
}

I don't see that as an improvement; you seem to be optimizing for the edge
case at the expense of the common case.

In particular, the explosion of TEST_* settings for Oracle all have to do with
the creation of a new user and tablespace; in the scenario where the user is
created for the test and deleted at its end, your suggestion doesn't seem to
make much sense -- normal connections don't deal with it, and the test
connection needs a "dba" connection to start from.

If I have misunderstood your suggestion, please clarify.

Thanks,
Shai.

[Michael Manfre]

On Tue, Jan 14, 2014 at 3:26 PM, Shai Berger <sh...@platonix.com> wrote:

The way testing currently works is by creating a throw-away database, running
the tests on that, and throwing it away. This means, among other things, that
you need settings for a database service and credentials for a user that can
create databases on this service. The common use-case is that this user is
your "main" database user, and the service is your main service. Your
suggestion, if I understand it correctly, gives the user two options:

1) Use separate credentials, and perhaps even a separate service, for testing;
this implies that the test database stays alive all the time.

As it stands, the privileges for running the tests are greater than those needed to run syncdb (or runserver), so I'm not sure what point you are trying to make. If you want to be able to use only one set of credentials to do everything, then you're going to use a user with enough privileges to do everything. This assumes you're not using Oracle and need to provide the slew of other configuration values, which should really be moved to OPTIONS.

2) Write configurations with repetitions or some other awkwardness, e.g.

default_db = {
        'ENGINE': 'this',
        'HOST': 'that',
        'NAME': 'the_other',
        'USER': 'me',
        'PASSWORD': 'my secret',
}

DATABASES = {
        'default' : default_db,
        'test_default' : dict(default_db,
                NAME='test_the_other',
                IS_TEST=True),
}

I don't see that as an improvement; you seem to be optimizing for the edge
case at the expense of the common case.

This is not what I envisioned. By separate aliases, I don't mean to just add more test aliases in with your existing settings. What I mean is that each database alias defined in a settings file should define a single connection configuration, not potentially two different configurations. If you need a different configuration for testing, put them in a separate configuration file.

 Example:

# settings.py - for local development

DATABASES = {

'default': {

'ENGINE': 'oracle',

'NAME': 'my_db',

'USER': 'my_user', # user can syncdb, but database exists

...

# 'CAN_USE_FOR_TESTS': False, # Protect database from unit test trampling

  }

}

# test_settings.py - for unit tests

DATABASES = {

'default': {

'ENGINE': 'oracle',

'NAME': 'test_my_db', # This was TEST_NAME

'USER': 'dba_user', # This is a user with proper credentials to drop a database

...

'OPTIONS': {

# backend specific settings belong here

'TEST_CREATE': True,

'TEST_TBLSPACE': 'test_',

'CHARSET': 'utf-8', # Or leave named as TEST_CHARSET

...

},

'CAN_USE_FOR_TESTS': True, # DB is safe to trample with tests

}

}

Regards,

Michael Manfre

[Shai Berger]

On Wednesday 15 January 2014 00:02:52 Michael Manfre wrote:
> On Tue, Jan 14, 2014 at 3:26 PM, Shai Berger <sh...@platonix.com> wrote:
> >
> > Your suggestion, if I understand it correctly, gives the user two options:
> >
> > 1) Use separate credentials, and perhaps even a separate service, for
> > testing;
> > this implies that the test database stays alive all the time.
>
> As it stands, the privileges for running the tests are greater than those
> needed to run syncdb (or runserver), so I'm not sure what point you are
> trying to make.

I did not understand your proposal correctly; I had thought you intended to
put the credentials for accessing the test database on equal footing with the
credentials for accessing the main database.

> This assumes you're not using Oracle and need to provide the
> slew of other configuration values, which should really be moved to
> OPTIONS.
>

I am still undecided about this one. We have backend-agnostic test-only
parameters (MIRROR and DEPENDENCIES, at least), we have backend-specific
parameters which apply to both testing and "production" (which usually go into
OPTIONS). I want the test-only parameters tucked away, and I'm not sure in
which of the above the test-only backend-specific parameters should go; but my
tendency is toward 'TEST'.

> > 2) Write configurations with repetitions or some other awkwardness, e.g.
> >

> > [example redacted]

> >
>
> This is not what I envisioned. By separate aliases, I don't mean to just
> add more test aliases in with your existing settings. What I mean is that
> each database alias defined in a settings file should define a single
> connection configuration, not potentially two different configurations. If
> you need a different configuration for testing, put them in a separate
> configuration file.
>
> Example:
>
> # settings.py - for local development
> DATABASES = {
> 'default': {

> ...
> # 'CAN_USE_FOR_TESTS': False, # Protect database from unit test trampling
> }
> }
>
>
> # test_settings.py - for unit tests
> DATABASES = {
> 'default': {
> 'ENGINE': 'oracle',
> 'NAME': 'test_my_db', # This was TEST_NAME

> 'USER': 'dba_user', # This is a user with credentials to drop a database

> ...
> 'OPTIONS': {
> # backend specific settings belong here
> 'TEST_CREATE': True,
> 'TEST_TBLSPACE': 'test_',
> 'CHARSET': 'utf-8', # Or leave named as TEST_CHARSET
> ...
> },
> 'CAN_USE_FOR_TESTS': True, # DB is safe to trample with tests
> }
> }
>

So, If I understand correctly (this time), your "test settings" are a strict
superset of your "local development" settings.

But I don't like the separation at all. It makes sense for some enterprise
environment, but not for a setting where each developer has their own database
installation. Making it in any concrete way will require the project to make
some decisions about the structure of the settings files, which we have avoided
for a very long time -- it is certainly out of scope of a little settings
clean-up. And some projects will just want to do something like 12factor,
where all credentials come from the environment rather than settings files, and
this is how you get different enterprisey settings.

If your suggestion is to make the separate files concretely supported by Django
-- e.g. use a different settings file, by default, for the test commands --
please flesh it out. I suspect I still don't quite follow where you're going
with this.

Shai.

[Shai Berger]

Hi,

I've put up https://github.com/django/django/pull/2400 for review. It makes
the changes discussed -- test settings go into a 'TEST' dictionary in the
database settings, with a deprecation warning for old settings.

While at it, I added (in a separate commit) a renaming of two Oracle-specific
settings -- because I don't like current names, and this is a good opportunity
to change them. At some future point, I want to make "CREATE_DB" available to
all backends, so this renaming is not just an Oracle issue.

There is documentation, but no tests -- database settings are impossible to
test automatically.

I want this to go in before the beta; it's a rather small PR. If there are no
comments, I'll probably commit it tomorrow or the day after.

Have fun,
Shai.

[Josh Smeaton]

Mentioned on the PR already, but writing some thoughts here too.

I like the change. Moving test settings into their own sub-dict makes a lot of sense. Missing are settings for creating the datafile and datafile size for Oracle though. I'm unsure if a separate patch should be submitted for those new settings or not. I'd really like if those new settings could be added for 1.7, to enable oracle test databases to be created by the test runner.

Currently, Oracle GIS can't be tested without first creating the test database (due to the datafile size). Similarly, the test runner can not create the database for Oracle RAC because of the datafile name.

Regards,

Josh

[Shai Berger]

Hi Josh,

Thanks for your comments.

The missing parameters are kinda a separate issue, with mush narrower
audience. Yes, I'd like to have them in 1.7 too, and I doubt I'll have time to
fix it myself -- so, PRs welcome.

Cheers,
Shai.