[Django] #17756: GeoDjango missing pyspatialite support
Django
-----------------------------+--------------------
Reporter: danols | Owner: nobody
Type: New feature | Status: new
Component: GIS | Version: SVN
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 1 | UI/UX: 0
-----------------------------+--------------------
Pyspatialite is a modified version of pysqlite that builds sqlite with
spatiliate support already in it. This module is easier to install
compared to pysqlite especially when utilizing a package installer such as
PIP; making GeoDjango support this package is relatively trivial.
Further Explanation:
It appears that the emergent way to run django sites is now
Virtualenv+PIP. Adding support for pyspatiallite allows one to setup
GeoDjango sqlite3 support utilizing PIP and hence allows for the creation
of requiremnts.txt which in turn makes the project that much more
portable. One can not use PIP to install pysqlite as by default it does
not enable loading of sqlite extensions.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: SVN
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by danols):
* owner: nobody => danols
* needs_better_patch: => 0
* status: new => assigned
* needs_tests: => 0
* needs_docs: => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:1>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by danols):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:2>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Comment (by anonymous):
the code is clear and is backwards compatible
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:3>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Changes (by juanpex):
* cc: juanpex (added)
Comment:
Replying to [comment:3 anonymous]:
> the code is clear and is backwards compatible
this is my comment.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:4>
Django
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Changes (by aaugustin):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:5>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Changes (by tmitchell):
* cc: taylor.mitchell@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:6>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Comment (by danols):
Can someone please mark it "Ready for checkin" ? I just applied the patch
on "Django-1.5.dev17930-py2.6-dev_r17930":
{{{
$ patch -p0 < patch-feb2312\@1646.diff
(Stripping trailing CRs from patch.)
patching file docs/ref/contrib/gis/install.txt
Hunk #1 succeeded at 66 (offset 2 lines).
Hunk #2 succeeded at 461 (offset 2 lines).
(Stripping trailing CRs from patch.)
patching file django/db/backends/sqlite3/base.py
Hunk #1 succeeded at 22 with fuzz 1.
Hunk #2 succeeded at 353 (offset 13 lines).
(Stripping trailing CRs from patch.)
patching file django/contrib/gis/db/backends/spatialite/base.py
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:7>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Comment (by claudep):
pyspatialite seems to suffer from build/install issues. See for example
those logs: http://code.activestate.com/pypm/pyspatialite/ [[BR]]
I was able to reproduce those errors on my system.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:8>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Comment (by claudep):
Note also that the following issue did not receive any feedback for two
months:[[BR]]
http://code.google.com/p/pyspatialite/issues/detail?id=6
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:9>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Comment (by danols):
Thanks for taking the time to review it; I've looked at the error logs and
the build fails as it is missing header files - unfortunately one has to
use the system package management to install those as pip won't work. Also
the branch is labeled experimental. I did an upgrade from 2.6 to this
experimental branch here and it build properly:
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-i686-2.6/src/module.o
build/temp.linux-i686-2.6/src/connection.o
build/temp.linux-i686-2.6/src/cursor.o
build/temp.linux-i686-2.6/src/cache.o
build/temp.linux-i686-2.6/src/microprotocols.o
build/temp.linux-i686-2.6/src/prepare_protocol.o
build/temp.linux-i686-2.6/src/statement.o
build/temp.linux-i686-2.6/src/util.o build/temp.linux-i686-2.6/src/row.o
build/temp.linux-i686-2.6/amalgamation/sqlite3.o
build/temp.linux-i686-2.6/amalgamation/spatialite.o
-L/Library/Frameworks/GEOS.framework/unix/lib
-L/Library/Frameworks/PROJ.framework/unix/lib -lgeos -lgeos_c -lproj -o
build/lib.linux-i686-2.6/pyspatialite/_spatialite.so
Successfully installed pyspatialite
Your second concern again results in header files missing; in this case
it's the geos package - see line amalgamation\spatialite.c:102:20: error:
geos_c.h: No such file or directory.
Both results are due to the fact PIP only does python packages and it
won't even complain if 'outside' requirements are missing, these
supporting header files would need to be installed either way to compile
sqlite3 with 'extension' support.
The patch ought to include a NOTE advising that one must install
supporting GEOS, GDAL, PROJ.4 files? Since these issues are not package
related after that it should be good for RFC. thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:10>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Changes (by claudep):
* needs_better_patch: 0 => 1
Comment:
Yes, please, provide any complementary instructions for system libraries
needed. What's strange with my system is that such libraries are
installed, as I'm running Geodjango/Postgis without problems. It might be
an issue with library paths.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:11>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by danols):
Will do; on a side note is your system Debian, Ubuntu? If so source files
and binary files are split up, i.e. libgeos libgeos-dev.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:12>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by claudep):
Yes, Debian. And you are right, the -dev packages are not installed, so
that should be part of the complementary instructions.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:13>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by danols):
Documentation updated and a pull requested created:
https://github.com/django/django/pull/198
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:14>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by claudep):
Thanks, we are close now. But I think that the docs are still not clear
enough about what steps are needed to install !SpatiaLite. The subsections
are currently:
* SQLite
* !SpatiaLite library (libspatialite) and tools (spatialite)
* pyspatialite
* pysqlite2
AFAIR, the pyspatialite method does not need any other step, right? In
that case, wouldn't it be better to structure the section this way:
* pyspatialite
* Installing from sources
* SQLite
* !SpatiaLite library (libspatialite) and tools (spatialite)
* pysqlite2
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:15>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by danols):
In all honestly when I was updating the docs I was thinking that the
install docs need to be overhauled. The install from source should be
removed and replaced with concise installs for specific database + linux
combo using PIP and system package installers.
!GeoDjango installation is overly confusing but does not need to be, I was
think along the lines:
* Requirements Overview
* Django + PostgreSQL + !GeoDjango Stack
* !Debian/Ubuntu
* CentOS
* ...
* Django + MySQL + !GeoDjango Stack
* !Debian/Ubuntu
* !CentOS
* ...
* !Django + !SQLite + !GeoDjango Stack
* !Debian/Ubuntu
* !CentOS
* ...
Thoughts ?
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:16>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by claudep):
Sure, feel free to propose a reorganisation of the install docs. However,
I'd prefer not mixing patches, and this could happen in a separate ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:17>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by akaariai):
I tested the pull request 198 with pyspatialite on Ubuntu 12.04. What I
did:
{{{
sudo apt-get install libgeos-dev
sudo pip install pyspatialite
./runtests --settings=test_spatialite.py gis
}}}
Result was a core dump. Output (but not the actual dump) attached.
Of course, I can't apply the PR as the above issue needs some resolution
first.
Additionally, please check the install.txt changes for whitespace errors,
lines longer than 80 chars and use spaces instead of tabs.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:18>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Changes (by reinout):
* cc: reinout@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:19>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Changes (by sfllaw):
* cc: sfllaw@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:20>
Django
Reporter: danols | Owner:
Type: New feature | Status: new
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
-----------------------------+-----------------------------------------
Changes (by danols):
* owner: danols =>
* status: assigned => new
* stage: Accepted => Someday/Maybe
Comment:
I no longer believe there is a need for this patch - installing through
Debian apt system all the requirements as stated by the docs bypasses all
of the PIP issues I have encountered and makes this not needed fix at
least for me; however feel free to change it.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:22>
Django
Reporter: danols | Owner: danols
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+------------------------------------
* easy: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:21>
Django
Reporter: danols | Owner:
Type: New feature | Status: new
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by anentropic):
* cc: anentropic (added)
Comment:
Just dropping in to say anything that can ease the pain of setting up
GeoDjango is much appreciated. I'm trying to do it primarily via Homebrew
and Pip... it'd be nice if it worked. The current instructions all lead to
various dead ends
I'm currently stuck here:
http://stackoverflow.com/questions/11093593/python-the-pysqlite-library-
does-not-support-c-extension-loading
If using `pip install pyspatialite` makes things more 'just work' then
PLEASE do it. It's not helpful to say oh yeah I got my install working
through Debian apt... it has zero value to users not on Debian.
Can a core dev please set this back to 'Accepted' as it had previously
been marked?
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:23>
Django
Reporter: danols | Owner:
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Comment (by anentropic):
Well after actually trying to apply the patch and go this route I think
there are more problems. Basically pyspatialite is broken.
https://code.google.com/p/pyspatialite/issues/detail?id=5
https://code.google.com/p/pyspatialite/issues/detail?id=9
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:24>
Django
Reporter: danols | Owner:
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Comment (by anonymous):
As a note, it's being/is fixed (at the moment, there is a new pre-release
package on pypi. after a bit of testing, I'll upgrade it to stable)
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:25>
Django
-----------------------------+-----------------------------------------
Reporter: danols | Owner:
Type: New feature | Status: new
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Comment (by ramiro):
Replying to [comment:23 anentropic]:
> Just dropping in to say anything that can ease the pain of setting up
GeoDjango is much appreciated. I'm trying to do it primarily via Homebrew
and Pip... it'd be nice if it worked. The current instructions all lead to
various dead ends
>
> I'm currently stuck here:
> http://stackoverflow.com/questions/11093593/python-the-pysqlite-library-
does-not-support-c-extension-loading
The answer to that SO question is the same as the one the Geodjango docs
have mentioned for some time:
https://docs.djangoproject.com/en/1.6/ref/contrib/gis/install/spatialite/#pysqlite2
> [...]
>
> Can a core dev please set this back to 'Accepted' as it had previously
been marked?
Having tried the pyspatialite path myself recently, even with the latest
development code, and getting the segmentation faults Anssi mentioned in
comment:18 back in Jul '12 when trying to execute syncdb seems to indicate
that it hasn't yet reached a point where we can:
* Support pypatialite by replacing the module used by the ''main'' SQLite
Django DB backend.
* Recommend it as the easiest path to get GIS support with SQLite.
and that the documented process of using platform-packages plus possibly
building pysqlite2 with a modified configuration still is the safest
strategy to reach success.
Because of this, I think the 'someday/maybe' tag is appropiate at this
point.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:26>
Django
-----------------------------+-----------------------------------------
Reporter: danols | Owner:
Type: New feature | Status: new
Component: GIS | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Comment (by anentropic):
I am still unable to install Spatialite. The solution in SO question and
GeoDjango docs mentioned above don't work for me. Full details here:
http://stackoverflow.com/questions/20505665/install-spatialite-for-python-
geodjango-on-os-x
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:27>
Django
-----------------------------+-----------------------------------------
Reporter: danols | Owner:
Component: GIS | Version: master
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
pysqlite 2.7.0+ can be installed using pip as C extension loading is now
[https://github.com/ghaering/pysqlite/commit/76f34850be582249d9048e6acb2993943b07d31b
enabled by default]. I believe that removes the motivation for this
ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/17756#comment:28>