[Django] #23152: Geodjango spatialite test database initialization very slow on some systems
Django
---------------------------------+----------------------
Reporter: tomlottermann@… | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.7-rc-2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 1 | UI/UX: 0
---------------------------------+----------------------
While testing some of my gis models, I noticed that the test database
setup ran really slowly (about 16 minutes on my harddisk).
I dug a little bit deeper into the code and noticed that the execution of
load_spatialite_sql() in contrib/gis/db/spatialite/creation.py went really
slowly. To be more precise the line:
{{{
cur.execute("SELECT InitSpatialMetaData()")
}}}
I found this issue here:
http://hub.qgis.org/issues/8340
This basically says that the InitSpatialMetaData() command is really slow
on some systems, because it executes in many transactions.
One could speed it up by using one of the following solutions:
1. pass 1 to the function. This makes (at least recent versions of
spatialite) it execute in a single transaction. I am not really sure for
which versions of spatialite this works.
{{{
cur.execute("SELECT InitSpatialMetaData(1)")
}}}
2. manually control the transaction:
{{{
cur.execute("BEGIN ;")
cur.execute("SELECT InitSpatialMetaData()")
cur.execute("COMMIT ;")
}}}
For people who have the test database outside the memory this might be a
huge problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/23152>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: tomlottermann@… | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Changes (by timo):
* needs_better_patch: => 0
* needs_tests: => 0
* version: 1.7-rc-2 => master
* easy: 1 => 0
* needs_docs: => 0
* type: Bug => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
From that link, it looks like this is only supported on Spatialite 4.1+ so
we may need some conditional logic to avoid breaking older versions.
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:1>
Django
Reporter: tomlottermann@… | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by cardoe):
I've implemented this and submitted it as part of the following pull
request: https://github.com/django/django/pull/3423 Would you accept the
same pull request against the 1.7 branch for a future stable bump?
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:2>
Django
Reporter: tomlottermann@… | Owner: cardoe
Type: Cleanup/optimization | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
* status: new => assigned
* owner: nobody => cardoe
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:3>
Django
Reporter: tomlottermann@… | Owner: cardoe
Component: GIS | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"57e40551e41eaf31ff96672beea9b1f732f2e331"]:
{{{
#!CommitTicketReference repository=""
revision="57e40551e41eaf31ff96672beea9b1f732f2e331"
Fixed #23152 -- Added support for transactional Spatialite metadata
initialization.
Thanks Doug Goldstein for the initial patch.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:4>
Django
Reporter: tomlottermann@… | Owner: cardoe
Type: Cleanup/optimization | Status: closed
Component: GIS | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"9c9f35ed27d1762f8da5ac0683d350952debf709"]:
{{{
#!CommitTicketReference repository=""
revision="9c9f35ed27d1762f8da5ac0683d350952debf709"
[1.7.x] Fixed #23152 -- Added support for transactional Spatialite
metadata initialization.
Thanks Doug Goldstein for the initial patch.
Backport of 57e40551e4 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:5>
Django
Reporter: tomlottermann@… | Owner: cardoe
Type: Cleanup/optimization | Status: closed
Component: GIS | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"e548d08f243335b4f11190417830a5f3027efd2a"]:
{{{
#!CommitTicketReference repository=""
revision="e548d08f243335b4f11190417830a5f3027efd2a"
Renamed SpatiaLite feature flag introduced in refs #23152.
Thanks Doug Goldstein for the suggestion.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23152#comment:6>