[Django] #21346: GeoDjango query repeatedly executed in loop

2 views

Skip to first unread message

Django

unread,

Oct 28, 2013, 6:23:01 PM10/28/13

to django-...@googlegroups.com

#21346: GeoDjango query repeatedly executed in loop
----------------------------+--------------------
Reporter: django@… | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------
I'm running Django 1.5.5 on OSX 10.7.5 with Python 2.7.5 and PostgreSQL
9.3.1 and PostGIS version 2.1.0"

{{{
"POSTGIS="2.1.0 r11822" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6
March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.9.1"
LIBJSON="UNKNOWN" TOPOLOGY RASTER"
}}}

I've discovered what appears to be a bug causing GeoDjango queries to be
repeatedly executed. What I'm doing here is seeing what locations touch
other locations. In this specific example, I'm querying France. My
database is populated from the [http://www.naturalearthdata.com/downloads
/10m-cultural-vectors/10m-admin-0-countries/ Natural Earth dataset].

{{{#!python
print datetime.datetime.now()
touching_locations =
Location.objects.filter(geometry__touches=Location.objects.get(name='France').geometry).values_list('name',
flat=True)
print type(touching_locations)
for i in range(10):
print datetime.datetime.now(), touching_locations
print datetime.datetime.now()
}}}

Output:

{{{
2013-10-28 16:09:14.958192
<class 'django.contrib.gis.db.models.query.GeoValuesListQuerySet'>
2013-10-28 16:09:14.974053 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:09:22.576670 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:09:30.089413 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:09:37.616438 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:09:45.119495 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:09:52.664080 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:10:00.197783 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:10:07.730839 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:10:15.270071 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:10:22.898417 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:10:30.429394
}}}

So, you can see that there's about 8 seconds between print statements.
This is obviously because it's repeatedly executing the query. Now, if I
force the query to be stored in a list, I see the performance I expect:

{{{#!python
print datetime.datetime.now()
touching_locations = [x for x in
Location.objects.filter(geometry__touches=Location.objects.get(name='France').geometry).values_list('name',
flat=True)]
print type(touching_locations)
for i in range(10):
print datetime.datetime.now(), touching_locations
print datetime.datetime.now()
}}}

{{{
2013-10-28 16:11:58.997853
<type 'list'>
2013-10-28 16:12:06.539672 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539702 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539714 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539724 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539734 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539744 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539755 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539765 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539775 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539784 [u'Belgium', u'Brazil', u'Germany', u'Spain',
u'Italy']
2013-10-28 16:12:06.539804
}}}

Now, you can see that the query is just called once, and performance is
much better.

--
Ticket URL: <https://code.djangoproject.com/ticket/21346>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Oct 30, 2013, 10:00:48 AM10/30/13

to django-...@googlegroups.com

#21346: GeoDjango query repeatedly executed in loop

Easy pickings: 0 | UI/UX: 0

--------------------------+--------------------------------------
Changes (by timo):

* status: new => closed
* needs_better_patch: => 0
* resolution: => invalid
* needs_tests: => 0
* needs_docs: => 0

Comment:

This looks like it's working just as documented:
https://docs.djangoproject.com/en/dev/topics/db/queries/#caching-and-
querysets