[Django] #28160: Can't start with django.contrib.gis.gdal.HAS_GDAL = False due to ImportError of GDALRaster
Django
ImportError of GDALRaster
-------------------------------------+-------------------------------------
Reporter: Tom | Owner: nobody
Kazimiers |
Type: Bug | Status: new
Component: GIS | Version: 1.10
Severity: Normal | Keywords: GIS, GDAL, Contrib,
Triage Stage: | Bug
Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Under some circumstances `django.contrib.gis.gdal.HAS_GDAL` is `False`.
This can be due to not having GDAL installed in the first place or by
overriding one's OS name (and thereby causing the test in
`django.contrib.gis.gdal.__init__.py` to result in an exception which in
turns causes `HAS_GDAL = False`).¹
Now with `HAS_GDAL = False`, I can't start Django anymore (neither the
shell or any other command), because I get the following error:
{{{
django.core.exceptions.ImproperlyConfigured: 'postgresql' isn't an
available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:
'mysql', 'oracle', 'postgresql', 'sqlite3'
Error was: No module named postgresql.base
}}}
Digging a bit deeper into this reveals this is actually an ImportError:
`Error was: cannot import name GDALRaster`. Using other
`django.contrib.gis` functionality should also work without GDAL (and did
so in the past).
The problem seems to be that in commit
`bbfad84dd980a97174c3b061a3d1b5f1373c380d`
([https://github.com/django/django/commit/bbfad84dd980a97174c3b061a3d1b5f1373c380d
#diff-9501b5a77c1985430f52d03aedb3e434R7 here]) a new import has been
added to `django/contrib/gis/db/backends/postgis/operations.py`, which
tries to import `GDALRaster`: `from django.contrib.gis.gdal import
GDALRaster`. This of course fails with `HAS_GDAL = False`, because
`GDALRaster` has never been defined when `HAS_GDAL` was set false due to
an (expected) exception in `django/django/contrib/gis/gdal/libgdal.py`,
which is imported before `from django.contrib.gis.gdal.raster.source
import GDALRaster` in `django/django/contrib/gis/gdal/__init__.py`.
So maybe the import in
`django/contrib/gis/db/backends/postgis/operations.py` and the use of
`GDALRaster` should be conditional and only run if `HAS_GDAL = True`. I
can provide a patch if wanted.
This is a problem in both Django 1.10 and 1.11.
¹ In my project I need to do the latter to disable GDAL support
completely, because loading the library conflicts with `pgmagick` on
Ubuntu and there is no other way to disable GDAL, but this shouldn't
matter here. It would be nice though to have a way to force `HAS_GDAL =
False`, because unlike said in
`django/django/contrib/gis/gdal/__init__.py`, setting
`GDAL_LIBRARY_PATH='/null/path'` doesn't work (the automatic OS checks
override this).
--
Ticket URL: <https://code.djangoproject.com/ticket/28160>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
ImportError of GDALRaster
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage:
Bug | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
A workaround for this is to run this code before the (postgis) database
driver is loaded (e.g. through an own custom database driver that just
sub-classes the original):
{{{
import django.contrib.gis.gdal
class GDALRasterMock(object):
pass
django.contrib.gis.gdal.GDALRaster = GDALRasterMock
}}}
¹ In my project I need to do the latter to disable GDAL support
completely, because loading the library conflicts with `pgmagick` on
Ubuntu and there is no other way to disable GDAL, but this shouldn't
matter here. It would be nice though to have a way to force `HAS_GDAL =
False`, because unlike said in
`django/django/contrib/gis/gdal/__init__.py`, setting
`GDAL_LIBRARY_PATH='/null/path'` doesn't work (the automatic OS checks
override this).
--
Comment (by Tom Kazimiers):
Add monkey-patch workaround
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:1>
Django
ImportError of GDALRaster
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
In Django 1.11, GDAL is now mandatory in order to use `django.contrib.gis`
(read https://docs.djangoproject.com/en/1.11/releases/1.11/#id1).
However, somewhere in the code, the import exception is swallowed, which
isn't nice at all. There should be at least a clear message that GDAL
cannot be imported.
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:2>
Django
ImportError of GDALRaster
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tom Kazimiers):
I see, thanks for the pointer to the Django 1.11 release notes that make
clear GDAL is required from 1.11 on to use `django.contrib.gis`. This
makes fixing the `HAS_GDAL = False` issue maybe not really worth it (given
that one can work around it) and I better figure out why using GDAL breaks
TIFF support in `pgmagick` (at least for me on Ubuntu). Fixing the
swallowed exception would be nice though.
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
Comment:
I think we can remove `HAS_GDAL` considering it's used only in tests and
instead use `runtests.py` to skip `gis_tests` unless a spatial backend is
being tested. Any other ideas or thoughts?
[https://github.com/django/django/pull/8451 PR] (one test still needs
fixing)
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"890537253cf235091816d27a5c2fb64943c8e34a" 89053725]:
{{{
#!CommitTicketReference repository=""
revision="890537253cf235091816d27a5c2fb64943c8e34a"
Made runtests.py run gis_tests only when using a GIS database backend.
This facilitates other changes like refs #28160.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Component: GIS | Version: 1.10
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"444cdf61314e420c10f713ba7a5fa084b0b2dfd5" 444cdf6]:
{{{
#!CommitTicketReference repository=""
revision="444cdf61314e420c10f713ba7a5fa084b0b2dfd5"
[1.11.x] Made runtests.py run gis_tests only when using a GIS database
backend.
This facilitates other changes like refs #28160.
Backport of 890537253cf235091816d27a5c2fb64943c8e34a from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Component: GIS | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"2dc3280254ae06ca1fe664abf55749fe12a699d4" 2dc3280]:
{{{
#!CommitTicketReference repository=""
revision="2dc3280254ae06ca1fe664abf55749fe12a699d4"
Fixed #28160 -- Prevented hiding GDAL exceptions when it's not installed.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Type: Bug | Status: closed
Severity: Normal | Resolution: fixed
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"68ebc240a47cb235a186a01f887db318e0a55eba" 68ebc24]:
{{{
#!CommitTicketReference repository=""
revision="68ebc240a47cb235a186a01f887db318e0a55eba"
[1.11.x] Fixed #28160 -- Prevented hiding GDAL exceptions when it's not
installed.
Backport of 2dc3280254ae06ca1fe664abf55749fe12a699d4 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Tom Kazimiers | Owner: nobody
Type: Bug | Status: closed
Severity: Normal | Resolution: fixed
Keywords: GIS, GDAL, Contrib, | Triage Stage: Accepted
Bug |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by GitHub <noreply@…>):
In [changeset:"4841fafb4421a215e854f68daebde2d4cfccfc3f" 4841fafb]:
{{{
#!CommitTicketReference repository=""
revision="4841fafb4421a215e854f68daebde2d4cfccfc3f"
[1.11.x] Refs #28160 -- Skipped a GeoManager test if not using a GIS
database backend.
The test errors if GDAL isn't installed.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28160#comment:10>