[Django] #25657: gevent-monkeypatching with GIS PointField causes intermittent traceback on shutdown
Django
shutdown
----------------------------+--------------------
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9b1
Severity: Normal | Keywords: gevent
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------
With the following model and a management command script that creates
several instances of this model - some with `point = None` - I get a
traceback on shutdown of the management command script. Note that the
instances are created/saved to the database perfectly fine. I also am
'''unable to reproduce''' without gevent.
{{{
from django.contrib.gis.db import models
class Provider(models.Model):
point = models.PointField(blank=True, null=True)
}}}
Management script:
{{{
from django.contrib.gis.geos import Point
from django.core.management import BaseCommand
from businesses.models import Provider
class Command(BaseCommand):
def handle(self, *args, **options):
for i in range(100):
Provider.objects.create(
point=Point(-123.329773, 48.407326)
)
Provider.objects.create(
point=None
)
}}}
Finally, the traceback:
{{{
(geos_exception)vagrant@vagrant-ubuntu-trusty-64:/vagrant/geos_exception$
./manage.py testcomm --settings=geos_exception.settings
Exception ignored in: <bound method WKBWriter.__del__ of
<django.contrib.gis.geos.prototypes.io.WKBWriter object at
0x7f7b62d85198>>
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-
packages/django/contrib/gis/geos/prototypes/io.py", line 137, in __del__
File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-
packages/django/contrib/gis/geos/libgeos.py", line 157, in __call__
File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-
packages/django/contrib/gis/geos/prototypes/threadsafe.py", line 52, in
__call__
File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-
packages/gevent/local.py", line 246, in __getattribute__
TypeError: 'NoneType' object is not callable
}}}
I am attached a tar.gz file containing the test project. Remember, this
issue is intermittent - I can run the management command a time or two
without traceback before seeing the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/25657>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9b1
Keywords: gevent | Triage Stage: Unreviewed
Has patch: 0 | Easy pickings: 0
UI/UX: 0 |
Changes (by jaddison):
* Attachment "geos_exception.tar.gz" added.
Sample project exhibiting the problem.
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9b1
Keywords: gevent | Triage Stage: Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
All of this is related to garbage collection, that's clear. `WKBWriter`/
`WKBReader` instances have a `__del__` method, so they are not part of the
normal Python garbage collection cycle (read
https://docs.python.org/3/reference/datamodel.html?highlight=__del__#object.__del__).
Looking at the traceback though, it seems that the error is happening in
`gevent` itself, that's why I doubt we'll able to fix this on Django's
side. Also note that these errors are more annoying than harmful.
#20903 also reported this (but without reproducible code).
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:1>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9b1
Keywords: gevent | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by jaddison):
New information: I have also seen this traceback, and I haven't even used
the model with the `PointField`:
{{{
Exception ignored in: <bound method Point.__del__ of <Point object at
0x7fb1c9e55a28>>
Traceback (most recent call last):
File
"/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/geometry.py",
line 125, in __del__
File
"/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py",
line 156, in __call__
File
"/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py",
line 160, in get_func
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2222, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 2155, in _find_spec
TypeError: 'NoneType' object is not iterable
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:2>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9b1
Keywords: gevent | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* stage: Unreviewed => Accepted
Comment:
I also got one of those exception today with Django 1.8.5/Python 3.4.
I think we should simply catch `TypeError` in the destructor methods and
ignore in case of exceptions.
So instead of current:
{{{
if self._ptr and capi:
capi.release_srs(self._ptr)
}}}
Let's write:
{{{
try:
capi.release_srs(self._ptr)
except TypeError:
pass
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:3>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 1.9
Keywords: gevent | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* version: 1.9b1 => 1.9
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:4>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: gevent | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/5778 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:5>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:6>
Django
shutdown
Reporter: jaddison | Owner: nobody
Type: Bug | Status: closed
Component: GIS | Version: 1.9
Severity: Normal | Resolution: fixed
Keywords: gevent | Triage Stage: Ready for checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz <claude@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"cd40d9e721eb4f5d05f57a74c6f1fd5d4aaef444" cd40d9e7]:
{{{
#!CommitTicketReference repository=""
revision="cd40d9e721eb4f5d05f57a74c6f1fd5d4aaef444"
Fixed #25657 -- Ignored exceptions when destroying geometry objects
Due to randomness of garbage collection with geometry objects, it's
easier to simply ignore AttributeError/TypeError generally raised when
parts of objects are already garbage-collected.
Thanks Sergey Fedoseev and Tim Graham for reviewing the patch.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25657#comment:7>