[Django] #21335: Bundled importlib some times won't work properly with python3.3
Django
-------------------------------+--------------------
Reporter: niwi | Owner: nobody
Type: Uncategorized | Status: new
Component: Python 3 | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
In one of my project some times (the most common case, run runserver,
change code, runsever reloads) and in some requests is raised strange
KeyError:
{{{
Traceback (most recent call last):
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/core/handlers/base.py", line 114, in get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/rest_framework/viewsets.py", line 78, in view
return self.dispatch(request, *args, **kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/views/decorators/csrf.py", line 57, in wrapped_view
return view_func(*args, **kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/rest_framework/views.py", line 399, in dispatch
response = self.handle_exception(exc)
File "/home/niwi/devel/greenmine-back/greenmine/base/monkey.py", line
23, in handle_exception
return super(APIView, self).handle_exception(exc)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/rest_framework/views.py", line 396, in dispatch
response = handler(request, *args, **kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/rest_framework/mixins.py", line 139, in partial_update
return self.update(request, *args, **kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/db/transaction.py", line 338, in inner
return func(*args, **kwargs)
File "/home/niwi/devel/greenmine-back/greenmine/base/api.py", line 28,
in update
return super().update(*args, **kwargs)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/rest_framework/mixins.py", line 132, in update
self.post_save(self.object, created=created)
File "/home/niwi/devel/greenmine-
back/greenmine/projects/userstories/api.py", line 90, in post_save
super().post_save(obj, created)
File "/home/niwi/devel/greenmine-
back/greenmine/base/notifications/api.py", line 25, in post_save
context["changed_fields_dict"] =
obj.get_changed_fields_dict(self.request.DATA)
File "/home/niwi/devel/greenmine-
back/greenmine/base/notifications/models.py", line 77, in
get_changed_fields_dict
field_dict.update(self._get_changed_field(field_name, data_value))
File "/home/niwi/devel/greenmine-
back/greenmine/base/notifications/models.py", line 126, in
_get_changed_field
old_value = self._get_changed_field_old_value(field_name, data_value)
File "/home/niwi/devel/greenmine-
back/greenmine/base/notifications/models.py", line 119, in
_get_changed_field_old_value
return (self.last_version and
self.last_version.field_dict.get(field_name, data_value) or None)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/reversion/models.py", line 177, in field_dict
object_version = self.object_version
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/reversion/models.py", line 164, in object_version
return list(serializers.deserialize(self.format, data,
ignorenonexistent=True))[0]
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/core/serializers/__init__.py", line 132, in deserialize
d = get_deserializer(format)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/core/serializers/__init__.py", line 111, in
get_deserializer
_load_serializers()
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/core/serializers/__init__.py", line 144, in
_load_serializers
register_serializer(format, BUILTIN_SERIALIZERS[format], serializers)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/core/serializers/__init__.py", line 69, in
register_serializer
module = importlib.import_module(serializer_module)
File "/home/niwi/.envs/greenmine/lib/python3.3/site-
packages/django/utils/importlib.py", line 36, in import_module
return sys.modules[name]
KeyError: 'django.core.serializers.pyyaml'
}}}
Saying code of bundled importlib...
{{{#!python
def import_module(name, package=None):
"""Import a module.
The 'package' argument is required when performing a relative import.
It
specifies the package to use as the anchor point from which to resolve
the
relative import to an absolute import.
"""
if name.startswith('.'):
if not package:
raise TypeError("relative imports require the 'package'
argument")
level = 0
for character in name:
if character != '.':
break
level += 1
name = _resolve_name(name[level:], package, level)
__import__(name)
return sys.modules[name]
}}}
... seems {{{__import__}}} is called with "django.core.serializers.pyyaml"
but it does nothing (not raises ImportError, but it should do it) and
sys.modules, obviously not contains "django.core.serializers.pyyaml".
(Note: I don't have pyyaml installed and I not use it on my project...)
I don't have deterministic way to reproduce this, because it hapens
sometimes using python3.3 and django1.6(alpha->rc1).
I think that it happens because a python2.7 importlib isn't work properly
with python3 and a simple solution is use python3 importlib if you running
django with python3:
https://github.com/niwibe/django/commit/2f73c41ed0e6e357b6710ae3b5bce4d7d55870cc
I know that importlib is now deprecated on master, but I think it should
fixed also in 1.5, 1.6 and master.
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/21335>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: new
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* needs_better_patch: => 0
* needs_docs: => 0
* type: Uncategorized => Bug
* needs_tests: => 0
* stage: Unreviewed => Accepted
Comment:
I think this is a safe change and I'm ready to do the backporting job.
However, I'd like to get fellow core devs approval before going forward.
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:1>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: new
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by aaugustin):
I'm OK with that.
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:2>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: closed
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Changes (by Claude Paroz <claude@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"3351e94ffa37d3824a49b687125ee4a430b69bbe"]:
{{{
#!CommitTicketReference repository=""
revision="3351e94ffa37d3824a49b687125ee4a430b69bbe"
Fixed #21335 -- Use importlib from python3 when using python3
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:3>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: closed
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Claude Paroz <claude@…>):
In [changeset:"e052ada0f6f21208d0f215d2f9184c4069ac1419"]:
{{{
#!CommitTicketReference repository=""
revision="e052ada0f6f21208d0f215d2f9184c4069ac1419"
[1.6.x] Fixed #21335 -- Use importlib from python3 when using python3
Backport of 3351e94ffa from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:4>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: closed
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Claude Paroz <claude@…>):
In [changeset:"1ef737416ab56a6becee36876769909a37541465"]:
{{{
#!CommitTicketReference repository=""
revision="1ef737416ab56a6becee36876769909a37541465"
[1.5.x] Fixed #21335 -- Use importlib from python3 when using python3
Backport of 3351e94ffa from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:5>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: closed
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Claude Paroz <claude@…>):
In [changeset:"ec0a3f5085219cd647d3e93841018987ce5384bf"]:
{{{
#!CommitTicketReference repository=""
revision="ec0a3f5085219cd647d3e93841018987ce5384bf"
[1.6.x] Removed relative usage of import_module
Python 3 version of importlib doesn't support this syntax.
Partial backport of fdd7a355bf. Refs #21335.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:6>
Django
Reporter: niwi | Owner: nobody
Type: Bug | Status: closed
Component: Python 3 | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Comment (by Claude Paroz <claude@…>):
In [changeset:"3b4b376bd1dde3cbe64cf8f1a4fb1756904af710"]:
{{{
#!CommitTicketReference repository=""
revision="3b4b376bd1dde3cbe64cf8f1a4fb1756904af710"
Revert "[1.5.x] Fixed #21335 -- Use importlib from python3 when using
python3"
This reverts commit 1ef737416ab56a6becee36876769909a37541465.
This would require changing some import_module calls (relative
syntax, see ec0a3f50852), too much for a stable release.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21335#comment:7>