Re: [Django] #14296: 'manage.py test' failing for apps that access read-only databases
Django
-----------------------------------+------------------------------------
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------
Changes (by oinopion):
* cc: tomek@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-----------------------------------+------------------------------------
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------
Comment (by RK):
What about if it were possible to specify in the database parameters that
the test runner should create the test databases using a different
database server config entirely? Eg:
{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'myproject',
'HOST': 'maindb',
# ... plus some other settings
},
'readonly': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'myproject',
'HOST': 'readonlydb',
'TEST_DATABASE': 'default'
# ... plus some other settings
}
}
}}}
This would trigger the test runner to create another test database on the
"default" server, for use with anything during the test which tries to
access the "readonly" database.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:10>
Django
-----------------------------------+------------------------------------
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------
Comment (by shai):
The Oracle backend supports a test parameter `CREATE_DB` that tells it not
to create the (tablespace, in this context equivalent to database) for
testing but rather use an existing one. It still tries to create tables,
but I guess the `managed=False` setting would take care of that in this
context.
I would like to make that parameter apply to all database backends.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:11>
Django
-----------------------------------+------------------------------------
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 1.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------
* cc: gregchapple1@… (added)
Comment:
As of b7aa7c4ab4372d2b7994d252c8bc87f77dd217ae you can use the `--keepdb`
parameter to use an existing database rather than creating a new one for
testing. This sounds more or less like what shai is describing above. It
will run any outstanding migrations, and flush the data, but ultimately
the database will be preserved at the end of the run.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:12>
Django
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 1.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by shaib):
* status: new => closed
* cc: shaib (added)
* resolution: => needsinfo
Comment:
6 years after a general workaround was suggested, and almost two years
since the last comment, I think that anybody who wants improvement on this
should specify exactly what they need and why.
In particular -- the combination of `--keepdb` and database routers
disallowing migrations on the read-only database will stop any automatic
schema changes, and using Luke's suggestions for test-cases should stop
data changes as well.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:13>
Django
-----------------------------------+------------------------------------
Type: Bug | Status: new
Component: Testing framework | Version: 1.2
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------
* status: closed => new
* resolution: needsinfo =>
Comment:
I'd like to add some more to this again as I'm having this problem with
Django 2.1.
I have some large Oracle DBs which are read only, and my app is providing
an API to. All my models are set to be `managed=False`, and I've set the
databases to have:
{{{
'TEST': {
'CREATE_DB': False,
'CREATE_USER': False,
'USER': env('DATABASE_USER_DEV'),
'PASSWORD': env('DATABASE_PASSWORD_DEV'),
}
}}}
However, running `python manage.py test` results in
`django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create
the django_migrations table (ORA-02000: missing ALWAYS keyword)` which
suggests to me that we need a "read-only" database option that will stop
all attempts to generate data on these connections.
For reference, the app uses a local sqlite or postgress DB for the data it
can store (and the standard Django apps) as the default database, but
there is no scope to be able to get write access to these external
databases, or have them duplicated for testing in any meaningful way (and
I understand Russell Keith-Mageen concern about live data in tests).
I'm not sure what the best solution here would be, for example:
* Using Create DB as False doesn't always mean you can't test the database
* Tests need to look for any table on the database that is managed, if
none found, treat as read-only (seems like an expensive option)
* Have a READ_ONLY test flag that would cause the standard tests to skip
some of the write based tests
The main output I want is to be able to run my tests (including the
standard ones from Django and other packages) without the early
termination of the test script because of the migrations exception.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:14>
Django
Reporter: kthhrv | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 1.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Jacob Walls):
* status: new => closed
* resolution: => needsinfo
Comment:
It's likely that the use case from comment:14 is solved through the
combination of setting `'TEST': {'MIGRATE': False}` (new in 3.1) and the
fix for #23273 (merged for 4.1) to skip creating the `django_migrations`
table.
Setting back to `needsinfo` in case someone has details to add around the
interaction of read-only databases with the test runner.
--
Ticket URL: <https://code.djangoproject.com/ticket/14296#comment:15>