[Django] #31002: Query on annotated spatial fields
Django
-------------------------------------------+------------------------
Reporter: VasileiosBouzas | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------------+------------------------
I am currently building a web app based on GeoDjango and I want to check
if a number of points lie within a multipolygon. To form the multipolygon,
I check which areas are selected and then aggregate them into one
multipolygon with the following query:
{{{
area = Area.objects.filter(id__in =
area_ids).aggregate(area=Union('geom'))['area']
print(area)
SRID=4326;MULTIPOLYGON (((5.014146 52.371687, 5.013973 52.371632, ...)))
}}}
Afterwards, I retrieved all the points from the **flows** queryset where
they are originally contained and annotate them to another **chains**
queryset under a new name (**'producer_geom'**):
{{{
subq = flows.filter(chains_id=OuterRef('pk'),
origin_role='producer')
chains = chains.annotate(pro_geom=
Subquery(subq.values('origin__administrative_location__geom'))
)
}}}
The result is the following:
{{{
print(chains.values_list('id', 'pro_geom'))
<QuerySet [(10804, <Point object at 0x7f656e633de0>), (10803, <Point
object at 0x7f656e649ab0>), (10802, <Point object at 0x7f656e649bc0>),
'...(remaining elements truncated)...']>
}}}
But when I try to filter the chains aftewards, to collect the ids of the
chains that the producer point lies with the area multipolygon,
{{{
chains = chains.filter(pro_geom__within=area)
}}}
I get the following error message:
{{{
Traceback (most recent call last):
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/rest_framework/viewsets.py", line 116, in view
return self.dispatch(request, *args, **kwargs)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/reversion/views.py", line 35, in do_revision_view
response = func(request, *args, **kwargs)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/rest_framework/views.py", line 495, in dispatch
response = self.handle_exception(exc)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/rest_framework/views.py", line 455, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/rest_framework/views.py", line 466, in raise_uncaught_exception
raise exc
File "/home/geofluxus/.local/lib/python3.6/site-
packages/rest_framework/views.py", line 492, in dispatch
response = handler(request, *args, **kwargs)
File "/home/geofluxus/Desktop/geoFluxus/repair/apps/utils/views.py",
line 36, in create
return self.post_get(request, **kwargs)
File
"/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py",
line 206, in post_get
queryset = self.filter_chain(queryset, filter_chains, keyflow)
File
"/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py",
line 356, in filter_chain
queryset = build_chain_filter(sub_filter, queryset, keyflow)
File
"/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py",
line 151, in build_chain_filter
chains = list(chains.values_list('id', flat=True))
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/query.py", line 182, in __iter__
for row in compiler.results_iter(chunked_fetch=self.chunked_fetch,
chunk_size=self.chunk_size):
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1052, in results_iter
results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch,
chunk_size=chunk_size)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 1087, in execute_sql
sql, params = self.as_sql()
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 489, in as_sql
where, w_params = self.compile(self.where) if self.where is not None
else ("", [])
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 405, in compile
sql, params = node.as_sql(self, self.connection)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/where.py", line 81, in as_sql
sql, params = compiler.compile(child)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 405, in compile
sql, params = node.as_sql(self, self.connection)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/where.py", line 81, in as_sql
sql, params = compiler.compile(child)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 405, in compile
sql, params = node.as_sql(self, self.connection)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/where.py", line 81, in as_sql
sql, params = compiler.compile(child)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/db/models/sql/compiler.py", line 405, in compile
sql, params = node.as_sql(self, self.connection)
File "/home/geofluxus/.local/lib/python3.6/site-
packages/django/contrib/gis/db/models/lookups.py", line 79, in as_sql
sql_params.extend(rhs_params)
AttributeError: 'tuple' object has no attribute 'extend'
}}}
Reproduced with Django v2.2.6 & v2.2.7
--
Ticket URL: <https://code.djangoproject.com/ticket/31002>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: VasileiosBouzas | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 2.2
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* stage: Unreviewed => Accepted
Comment:
Likely caused by our badly defined `Expression/Lookup.as_sql` signature
where we sometimes return `params` as a `list` and sometimes as a `tuple`.
Given `Subquery.as_sql` returns them as a tuple, through `Query.as_sql`,
we likely want to adjust `GISLookup.as_sql` here.
Could you give the following patch a try Vasileios
{{{#!diff
diff --git a/django/contrib/gis/db/models/lookups.py
b/django/contrib/gis/db/models/lookups.py
index f2af05d9c1..e9c3fc55dd 100644
--- a/django/contrib/gis/db/models/lookups.py
+++ b/django/contrib/gis/db/models/lookups.py
@@ -75,9 +75,9 @@ class GISLookup(Lookup):
return connection.ops.gis_operators[self.lookup_name]
def as_sql(self, compiler, connection):
- lhs_sql, sql_params = self.process_lhs(compiler, connection)
+ lhs_sql, lhs_params = self.process_lhs(compiler, connection)
rhs_sql, rhs_params = self.process_rhs(compiler, connection)
- sql_params.extend(rhs_params)
+ sql_params = tuple(lhs_params) + tuple(rhs_params)
template_params = {'lhs': lhs_sql, 'rhs': rhs_sql, 'value': '%s',
**self.template_params}
rhs_op = self.get_rhs_op(connection, rhs_sql)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31002#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Vasileios Bouzas | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: GIS | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* status: new => assigned
* owner: nobody => Simon Charette
--
Ticket URL: <https://code.djangoproject.com/ticket/31002#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Vasileios Bouzas | Owner: Simon
| Charette
Type: Bug | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/31002#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Vasileios Bouzas | Owner: Simon
| Charette
Type: Bug | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* version: 2.2 => master
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/31002#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Vasileios Bouzas | Owner: Simon
| Charette
Component: GIS | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"0290e01d5a202507e9a71294ac9cb31725c951bb" 0290e01]:
{{{
#!CommitTicketReference repository=""
revision="0290e01d5a202507e9a71294ac9cb31725c951bb"
Fixed #31002 -- Fixed GIS lookups crash against a subquery annotation.
Thanks Vasileios Bouzas for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31002#comment:5>