[Django] #34302: SpatialReference.srid incorrectly assumes first AUTHORITY value to be projection SRID
Django
projection SRID
----------------------------------------+------------------------
Reporter: Stefan Brand | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 4.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
----------------------------------------+------------------------
== Problem
When reading data into our PostGIS database, we noticed strange locations
for our geometries. The data was supposed to be `EPSG:31287`, but
investigation showed that Django was assuming `EPSG:6312`. This led to
wrong EWKTs when saving geometries to PostGIS.
It turns out that the dataset did not have any SRID stored for the
projection, just for the DATUM. Django assumed the DATUM's AUTHORITY value
to be the SRID of the dataset's projection. This '''contradicts the
[https://docs.djangoproject.com/en/4.1/ref/contrib/gis/gdal/#django.contrib.gis.gdal.SpatialReference.srid
documentation]''' for `SpatialReference.srid`:
> Returns the SRID of top-level authority, or None if undefined.
Since the layer's `srid` cannot be set to a different value, the only
workaround is to set the `srid` for each `OGRGeometry` individually.
== Current Result
[https://github.com/django/django/blob/0265b1b49ba10f957abfd1311d0bae0ecefc3111/django/contrib/gis/gdal/srs.py#L208
`SpatialReference.srid`] calls `int(self.attr_value("AUTHORITY", 1))`,
which returns `6312` (wrong SRID).
== Expected Result
`SpatialReference.srid` should call `auth_code(target=None)` to get the
top-level SRID.
Django should not assume a wrong SRID if the projection does not have a
SRID. For example, QGIS reads the whole spatial reference information,
correctly projects the data, but displays "Unknown CRS" because it does
not assume any AUTHORITY value to be the dataset SRID.
== Steps to reproduce
{{{
#!python
from django.contrib.gis.gdal import SpatialReference
WKT = """PROJCRS["MGI / Austria Lambert",
BASEGEOGCRS["MGI",
DATUM["Militar-Geographische Institut",
ELLIPSOID["Bessel 1841",6377397.155,299.1528128,
LENGTHUNIT["metre",1]],
ID["EPSG",6312]],
PRIMEM["Greenwich",0,
ANGLEUNIT["Degree",0.0174532925199433]]],
CONVERSION["unnamed",
METHOD["Lambert Conic Conformal (2SP)",
ID["EPSG",9802]],
PARAMETER["Latitude of false origin",47.5,
ANGLEUNIT["Degree",0.0174532925199433],
ID["EPSG",8821]],
PARAMETER["Longitude of false origin",13.3333333333333,
ANGLEUNIT["Degree",0.0174532925199433],
ID["EPSG",8822]],
PARAMETER["Latitude of 1st standard parallel",49,
ANGLEUNIT["Degree",0.0174532925199433],
ID["EPSG",8823]],
PARAMETER["Latitude of 2nd standard parallel",46,
ANGLEUNIT["Degree",0.0174532925199433],
ID["EPSG",8824]],
PARAMETER["Easting at false origin",400000,
LENGTHUNIT["metre",1],
ID["EPSG",8826]],
PARAMETER["Northing at false origin",400000,
LENGTHUNIT["metre",1],
ID["EPSG",8827]]],
CS[Cartesian,2],
AXIS["easting",east,
ORDER[1],
LENGTHUNIT["metre",1,
ID["EPSG",9001]]],
AXIS["northing",north,
ORDER[2],
LENGTHUNIT["metre",1,
ID["EPSG",9001]]]]"""
geodjango_srs = SpatialReference(WKT)
geodjango_srs.validate() # raises for invalid SRS
print(geodjango_srs.attr_value("AUTHORITY", 1))
print(geodjango_srs.auth_code(target=None))
}}}
> 6312
> None
--
Ticket URL: <https://code.djangoproject.com/ticket/34302>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
projection SRID
Reporter: Stefan Brand | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz):
* version: 4.1 => dev
* stage: Unreviewed => Accepted
Comment:
Thanks for the report. Would you like to prepare a pull request for
Django?
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:1>
Django
projection SRID
------------------------------+------------------------------------
Reporter: Stefan Brand | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------
Comment (by Stefan Brand):
Thanks for asking, yes, I would like that. I hope I can tackle it this
week. If I don't find time, someone else feel free to create a PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:2>
Django
projection SRID
Reporter: Stefan Brand | Owner: Rahmat Faisal
Type: Bug | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Rahmat Faisal):
* owner: nobody => Rahmat Faisal
* status: new => assigned
Comment:
gonna look
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:3>
Django
projection SRID
Reporter: Stefan Brand | Owner: (none)
Type: Bug | Status: new
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+------------------------------------
* owner: Rahmat Faisal => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:4>
Django
projection SRID
Reporter: Stefan Brand | Owner: Stefan Brand
Type: Bug | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Stefan Brand):
* owner: nobody => Stefan Brand
* status: new => assigned
Comment:
While applying the change, I got a failing test. It turns out that
GeoDjango has a different behaviour than the GDAL Python bindings.
Compare:
{{{
#!python
from django.contrib.gis.gdal import SpatialReference
from osgeo import ogr
WEB_MERCATOR = """PROJCS["WGS 84 / Pseudo-Mercator",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Mercator_1SP"],
PARAMETER["central_meridian",0],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["Easting",EAST],
AXIS["Northing",NORTH],
EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0
+x_0=0 +y_0=0 +k=1 +units=m +nadgrids=@null +wktext +no_defs"],
AUTHORITY["EPSG","3857"]]"""
geodjango_srs = SpatialReference(WEB_MERCATOR)
geodjango_srs.validate() # raises for invalid SRS
print(geodjango_srs.attr_value("AUTHORITY", 1))
print(geodjango_srs.auth_code(target=None))
ogr_srs = ogr.osr.SpatialReference(WEB_MERCATOR)
if ogr_srs.Validate(): raise
print(ogr_srs.GetAttrValue("AUTHORITY", 1))
print(ogr_srs.GetAuthorityCode(None))
}}}
> 3857
> None
> 3857
> 3857
The reason is that
[https://github.com/django/django/blob/fd21f82aa82b0d75a161f618ef944ebe0923e0ab/django/utils/encoding.py#L91-L108
`django/utils/force_bytes`] turns `None` into the byte string `b'None'`.
Since `ctypes.c_char_p` also accepts `None`, we can bypass `force_bytes`
if `target is None`.
I will do some more testing and push a patch tomorrow.
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:3>
Django
projection SRID
Reporter: Stefan Brand | Owner: Stefan
| Brand
Type: Bug | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* has_patch: 0 => 1
* stage: Accepted => Ready for checkin
Comment:
[https://github.com/django/django/pull/16564 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:4>
Django
projection SRID
-------------------------------------+-------------------------------------
Reporter: Stefan Brand | Owner: Stefan
| Brand
Type: Bug | Status: assigned
Component: GIS | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"d77762de038d1ab46cdcda2b7202d36c80956e25" d77762d]:
{{{
#!CommitTicketReference repository=""
revision="d77762de038d1ab46cdcda2b7202d36c80956e25"
Refs #34302 -- Fixed SpatialReference.auth_name()/auth_code() when target
is None.
force_bytes() turns None into the byte string b"None". Since
ctypes.c_char_p() also accepts None, we can bypass force_bytes() if
target is None.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:5>
Django
projection SRID
-------------------------------------+-------------------------------------
Reporter: Stefan Brand | Owner: Stefan
| Brand
Component: GIS | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"eacf6b73d8eace004f840bd9b80c8c671caab9da" eacf6b73]:
{{{
#!CommitTicketReference repository=""
revision="eacf6b73d8eace004f840bd9b80c8c671caab9da"
Fixed #34302 -- Fixed SpatialReference.srid for objects without top-level
authority.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:6>
Django
projection SRID
-------------------------------------+-------------------------------------
Reporter: Stefan Brand | Owner: Stefan
| Brand
Type: Bug | Status: closed
Component: GIS | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"efcc0f25a77120f728379a594af3e256e771b7f8" efcc0f2]:
{{{
#!CommitTicketReference repository=""
revision="efcc0f25a77120f728379a594af3e256e771b7f8"
[4.2.x] Fixed #34302 -- Fixed SpatialReference.srid for objects without
top-level authority.
Backport of eacf6b73d8eace004f840bd9b80c8c671caab9da from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:8>
Django
projection SRID
-------------------------------------+-------------------------------------
Reporter: Stefan Brand | Owner: Stefan
| Brand
Type: Bug | Status: closed
Component: GIS | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"341f33ed157809906b7c4b4ee59d8f11e84462ee" 341f33ed]:
{{{
#!CommitTicketReference repository=""
revision="341f33ed157809906b7c4b4ee59d8f11e84462ee"
[4.2.x] Refs #34302 -- Fixed SpatialReference.auth_name()/auth_code() when
target is None.
force_bytes() turns None into the byte string b"None". Since
ctypes.c_char_p() also accepts None, we can bypass force_bytes() if
target is None.
Backport of d77762de038d1ab46cdcda2b7202d36c80956e25 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34302#comment:7>