[Django] #36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Type:
| Cleanup/optimization
Status: new | Component: GIS
Version: dev | Severity: Normal
Keywords: gdal | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
I'd like to propose enabling lazy loading of the GDAL library in
`django.contrib.gis.gdal`, matching the existing behavior of
`django.contrib.gis.geos` with the GEOS library.

Currently, calling `django.setup()` on projects using GeoDjango fails if
GDAL is not installed, even when GDAL functionality is not being used.
This creates an inconsistency in the GeoDjango framework where the GEOS
library supports lazy loading (implemented ~10 years ago) but GDAL does
not. The proposed change would implement lazy loading for the GDAL module
similar to the existing GEOS implementation, ensuring the GDAL library is
only loaded when actually accessed or used.

I have a working proof-of-concept in a
[https://github.com/joshuadavidthomas/django/tree/lazy-gdal branch] on my
fork of the Django repo ready for review
([https://github.com/django/django/compare/main...joshuadavidthomas:django
:lazy-gdal diff]).

The implementation follows the same approach as the original GEOS lazy
loading commit, wrapping the library loading in `SimpleLazyObject` and
converting function prototypes to use lazy loading. The public API remains
unchanged (everything imported in `django/contrib/gis/gdal/__init__.py`
continues to work as before), though the imports related to the GDAL
version from `django.contrib.gis.gdal.libgdal` now use lazy evaluation.
The existing test suite passes without modification, and I've added new
tests specifically for the lazy loading behavior. Documentation updates
have not been included yet as I wasn't sure if that was needed for a
change like this.

I have created both a [https://forum.djangoproject.com/t/proposal-lazy-
loading-for-django-contrib-gis-gdal/41198 forum post] and an
[https://github.com/django/new-features/issues/42 issue on django/new-
features]. Both have only been open for a few days, so not a ton of time
for people to see them. However, the initial reception has been positive,
with the forum post receiving several likes and the new-features issue
getting only thumbs-up reactions.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`

--------------------------------------+------------------------------------
Reporter: Josh Thomas | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: GIS | Version: dev

Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Carlton Gibson):

* stage: Unreviewed => Accepted

Comment:

Thanks Josh, sounds good.

You said something about having a branch, so you can assign yourself and
open a PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:1>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`

-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned

Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Josh Thomas):

* owner: (none) => Josh Thomas
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:2>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Josh Thomas):

* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/19529 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:3>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_docs: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:4>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Josh Thomas):

* needs_docs: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:5>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Josh Thomas):

I've updated the release note documentation updates from 6.0 to 6.1, as I
assume due to the feature freeze this PR will not be making it in to 6.0.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:6>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Brad Stricherz):

* needs_better_patch: 0 => 1

Comment:

I reviewed PR #19529 and ran the new GDAL lazy-loading tests locally. With
PostGIS configured and GDAL missing, django.setup() no longer errors, and
the tests in gis_tests.gdal_tests.tests pass.

There is a missing import in django/contrib/gis/gdal/driver.py:


{{{
NameError: name 'classproperty' is not defined
}}}


This can be fixed by adding:


{{{
from django.utils.functional import classproperty
}}}



Once that is corrected, the patch should be ready for checkin. Marking as
Patch needs improvement until the import is added.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:7>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Josh Thomas):

* needs_better_patch: 1 => 0

Comment:

Ah! Thanks for the heads up. That came up in a review from @benjaoming and
I accepted the suggestion without thinking about the import. I've fixed
the import and rebased to the most recent changes on main.

Replying to [comment:7 Brad Stricherz]:

> I reviewed PR #19529 and ran the new GDAL lazy-loading tests locally.
With PostGIS configured and GDAL missing, django.setup() no longer errors,
and the tests in gis_tests.gdal_tests.tests pass.
>
> There is a missing import in django/contrib/gis/gdal/driver.py:
>
>
> {{{
> NameError: name 'classproperty' is not defined
> }}}
>
>
> This can be fixed by adding:
>
>
> {{{
> from django.utils.functional import classproperty
> }}}
>
>
>
> Once that is corrected, the patch should be ready for checkin. Marking
as Patch needs improvement until the import is added.
--

Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:8>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

Comment:

I think we need to add a system check to ensure misconfiguration is still
caught, and let laziness be achieved if desired by skipping said check.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:9>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

I'm skeptical that a system check is appropriate. The system check for
Pillow only runs if you have an ImageField in your model. Similarly, I
think it makes sense that a message about GDAL wouldn't appear until you
use a function that requires it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:10>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

> The system check for Pillow only runs if you have an ImageField in your
model.

I had in mind something just like that, a check that runs only on
`BaseSpatialField`. Would that be appropriate?
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:11>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Josh Thomas):

I'm not opposed to a system check. I can see the value from an API
stability perspective if people have been relying on eager failure to
catch misconfigurations. (Though I gotta admit it makes me think a little
bit of [https://xkcd.com/1172 this XKCD] 😆.)

GEOS has been lazy-loaded for ~10 years without a check, meaning
misconfigured GEOS installations have only been caught at runtime when
geometry fields are actually used and I haven’t really seen complaints
about that (though maybe I’ve just missed them!).

The question is: is a check needed because GDAL genuinely needs different
treatment than GEOS, or is this just noticing the gap while touching GDAL
code? I admit my ignorance on the details on the differences between the
two libraries here.

What would be the preference?

1. Add it to `BaseSpatialField` which would check both GEOS + GDAL
2. Add it to `RasterField` only, which uses GDAL but not GEOS, and would
more mirror the Pillow/`ImageField` pattern
3. Do nothing and let GDAL follow the same lazy pattern as GEOS
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:12>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

I naively assumed importing gdal at the top of
`django.contrib.gis.db.models.fields` would raise an exception, but I
guess the point of that change is that it doesn't. So at what point would
an exception be raised? How does it compare to GEOS? Also consider that
it's possible to use aspects of GeoDjango that rely on GEOS and GDAL
without using the model fields.

I'm not sure if there's any reason to treat GEOS + GDAL differently. My
impression is that this is a new idea that Jacob had. If we do add it, the
check for both could be authored as a separate commit.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:13>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

I doubt anything is more logically special about GDAL over GEOS in this
regard, but we do have this
[https://docs.djangoproject.com/en/5.2/releases/1.11/#id1 language] from
the Django 1.1 release saying it's required to use GeoDjango:

> To simplify the codebase and because it’s easier to install than when
contrib.gis was first released, GDAL is now a required dependency for
GeoDjango. In older versions, it’s only required for SQLite.

The complicated mechanism for detecting whether GDAL was present was
removed in #28160.

Is GEOS any different? I don't know. It sounds like it's required from
reading
[https://docs.djangoproject.com/en/5.2/ref/contrib/gis/install/#requirements
GeoDjango requirements]. I don't get the sense that either of them are
optional.

A system check to catch misconfiguration is in the spirit of the
contrib.postgres check from #32770. I still think I would miss it if the
current proposal takes away the eager imports.

>Also consider that it's possible to use aspects of GeoDjango that rely on
GEOS and GDAL without using the model fields.

I agree -- this should probably just be a system check registered at the
`contrib.gis` app level, not field level.

----
Given that we didn't have a check for GEOS, I can relent on blocking this
work over it if no one else thinks it's important. I only have the
anecdotal evidence that it was only ever the GDAL installation that gave
me trouble. (Thanks, HomeBrew?🍺)

Also anecdotal, but on the project forum for the software I used to work
on, the user support requests
[https://community.archesproject.org/search?q=GDAL_LIBRARY_PATH also
seemed to be about 2x for configuring GDAL] vs GEOS, but I acknowledge
that's a supremely small sample size...

I think a system check at the app level makes sense to ship in the same
release we are doing this other work. If Josh is up for it, I think it
makes sense to tag along now as a `Refs #...` commit. Otherwise I can pick
it up.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:14>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jacob Walls):

(We might even say that not detecting misconfiguration would re-cause
#28160.)
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:15>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Old description:

New description:

I'd like to propose enabling lazy loading of the GDAL library in
`django.contrib.gis.gdal`, matching the existing behavior of
`django.contrib.gis.geos` with the GEOS library.

Currently, calling `django.setup()` on projects using GeoDjango fails if
GDAL is not installed, even when GDAL functionality is not being used.
This creates an inconsistency in the GeoDjango framework where the GEOS

library supports lazy loading (implemented in 2015 in
61d09e61f5747d7a70268ca8d5e770486877500b) but GDAL does not. The proposed

History: GDAL was initially (inadvertently) required before it was made
optional in #5433 and then was made required in #26753 in order to simply
the code base.

--
Comment (by Tim Graham):

I don't know about that proposal. It seems strange to have to silence
warnings for "optional dependencies" that may not be needed. I would like
to see my questions in comment:13 answered in order to understand how a
"missing GDAL" error would be surfaced to a user.

By the way, despite the lazy loading commit for GEOS, it's
[https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/geolibs
/#geospatial-libraries still listed as a required dependency]. I guess not
much of GeoDjango is usable without it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:16>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

I think I'm missing your meaning, Tim -- why do you refer to GDAL as
optional? Your link docs both libraries as required, and I'm not aware of
test configurations supporting the idea that it's optional.

Josh is writing [https://github.com/joshuadavidthomas/django-language-
server a language server] where he wants to be able defer the imports.
That's not a typical use case, so I think it's fair to ask his project to
silence a system check (I'm assuming a language server would skip all
checks anyway for speed). I don't think that makes GDAL "optional" for
most `contrib.gis` installs.

> I would like to see my questions in comment:13 answered in order to
understand how a "missing GDAL" error would be surfaced to a user.

The PR has the effect of deferring any `ImproperlyConfigured` exception
from startup to the first use of GDAL, presumably in a view. For a site-
wide context processor, every view would fail, so you'd have a reasonable
chance of catching it immediately (indeed, the test project I'm using does
this). But where GIS is used more sparingly (e.g. just in an admin
widget), you wouldn't. The failure would be deferred until someone logged
into the admin. Deferring the error to an indeterminate time seems like an
unwelcome change. Am I understanding your question?
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:17>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* cc: Claude Paroz (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:18>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

Here's the deferred [https://dpaste.com/634FN7QKE traceback] when visiting
the admin (editing an instance having a value in a geometry field).
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:19>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

So the idea is to continue documenting GEOS and GDAL and required
dependencies but explaining that for some niche use cases, you may able to
use parts of GeoDjango without them? I suppose it's okay.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:20>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Claude Paroz):

I think I can live with that. But still, having a way to find out if
GeoDjango requirements are installed before reaching the loading error
would be a nice addition, maybe in the form of a management command? This
should be an other ticket, probably.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:21>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

> This should be an other ticket, probably.

Happy to manage this however folks like, but my proposal for the system
check was intended to meet that need. This would be roughly analogous to
what we do for Pillow, just registered for `contrib.gis` users. We could
take it as a `Refs #...` commit or a new ticket, but I'd want to have it
at the same time.

> explaining that for some niche use cases, you may able to use parts of
GeoDjango without them?

I don't think we need any documentation updates, this is just a cleanup
ticket to add flexibility for some tooling.
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:22>

[Django]

#36441: Add lazy loading support for GDAL library in `django.contrib.gis.gdal`
-------------------------------------+-------------------------------------
Reporter: Josh Thomas | Owner: Josh
Type: | Thomas
Cleanup/optimization | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: gdal | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jacob Walls):

> So at what point would an exception be raised? How does it compare to
GEOS?

I rechecked this, and GEOS isn't lazy-loaded after all, I don't think.
This is the traceback I get on startup when setting `GEOS_LIBRARY_PATH =
"/dev/null"`:

{{{#!py
File "/Users/jwalls/django/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jwalls/django/django/apps/registry.py", line 116, in
populate
app_config.import_models()
~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Users/jwalls/django/django/apps/config.py", line 269, in
import_models
self.models_module = import_module(models_module_name)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File
"/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/__init__.py",
line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
File "<frozen importlib._bootstrap>", line 488, in
_call_with_frames_removed
File "/Users/jwalls/django/django/contrib/auth/models.py", line 5, in
<module>
from django.contrib.auth.base_user import AbstractBaseUser,
BaseUserManager
File "/Users/jwalls/django/django/contrib/auth/base_user.py", line 43,
in <module>
class AbstractBaseUser(models.Model):
...<120 lines>...
)
File "/Users/jwalls/django/django/db/models/base.py", line 146, in
__new__
new_class.add_to_class("_meta", Options(meta, app_label))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jwalls/django/django/db/models/base.py", line 394, in
add_to_class
value.contribute_to_class(cls, name)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/Users/jwalls/django/django/db/models/options.py", line 245, in
contribute_to_class
self.db_table, connection.ops.max_name_length()
^^^^^^^^^^^^^^
File "/Users/jwalls/django/django/utils/connection.py", line 15, in
__getattr__
return getattr(self._connections[self._alias], item)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/Users/jwalls/django/django/utils/connection.py", line 62, in
__getitem__
conn = self.create_connection(alias)
File "/Users/jwalls/django/django/db/utils.py", line 196, in
create_connection
backend = load_backend(db["ENGINE"])
File "/Users/jwalls/django/django/db/utils.py", line 116, in
load_backend
return import_module("%s.base" % backend_name)
File
"/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/__init__.py",
line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/Users/jwalls/django/django/contrib/gis/db/backends/postgis/base.py",
line 16, in <module>
from .adapter import PostGISAdapter
File
"/Users/jwalls/django/django/contrib/gis/db/backends/postgis/adapter.py",
line 6, in <module>
from django.contrib.gis.geos import GEOSGeometry
File "/Users/jwalls/django/django/contrib/gis/geos/__init__.py", line 6,
in <module>
from .collections import ( # NOQA
...<4 lines>...
)
File "/Users/jwalls/django/django/contrib/gis/geos/collections.py", line
7, in <module>
from django.contrib.gis.geos.geometry import GEOSGeometry,
LinearGeometryMixin
File "/Users/jwalls/django/django/contrib/gis/geos/geometry.py", line
18, in <module>
from django.contrib.gis.geos.prototypes.io import ewkb_w, wkb_r,
wkb_w, wkt_r, wkt_w
File "/Users/jwalls/django/django/contrib/gis/geos/prototypes/io.py",
line 249, in <module>
class WKBWriter(IOBase):
...<67 lines>...
wkb_writer_set_include_srid(self.ptr, bool(include))
File "/Users/jwalls/django/django/contrib/gis/geos/prototypes/io.py",
line 253, in WKBWriter
geos_version = geos_version_tuple()
File "/Users/jwalls/django/django/contrib/gis/geos/libgeos.py", line
175, in geos_version_tuple
return get_version_tuple(geos_version().decode())
~~~~~~~~~~~~^^
File "/Users/jwalls/django/django/contrib/gis/geos/libgeos.py", line
170, in geos_version
return lgeos.GEOSversion()
^^^^^^^^^^^^^^^^^
File "/Users/jwalls/django/django/utils/functional.py", line 251, in
inner
self._setup()
~~~~~~~~~~~^^
File "/Users/jwalls/django/django/utils/functional.py", line 404, in
_setup
self._wrapped = self._setupfunc()
~~~~~~~~~~~~~~~^^
File "/Users/jwalls/django/django/contrib/gis/geos/libgeos.py", line 62,
in load_geos
_lgeos = CDLL(lib_path)
File
"/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/ctypes/__init__.py",
line 390, in __init__
self._handle = _dlopen(self._name, mode)
~~~~~~~^^^^^^^^^^^^^^^^^^
OSError: dlopen(/dev/null, 0x0006): tried: '/dev/null' (not a file),
'/System/Volumes/Preboot/Cryptexes/OS/dev/null' (no such file),
'/dev/null' (not a file)
}}}

So the effect of the PR would be to hide failures only for GDAL, rather
than to bring the behavior to parity with GEOS.

Josh, do you agree that a simple system check for a working registration
of GDAL registered by `contrib.gis` would be enough to move this forward?
--
Ticket URL: <https://code.djangoproject.com/ticket/36441#comment:23>