[Django] #17917: Pickling queryset with annotations on related fields with callable defaults fails
Django
fails
---------------------------------+-----------------------------------------
Reporter: jensadne@… | Owner: nobody
Type: Bug | Status: new
Component: ORM aggregation | Version: SVN
Severity: Normal | Keywords: queryset,pickle,aggregation
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+-----------------------------------------
Given the following models:
{{{
#!python
class Person(models.Model):
name = models.CharField(max_length=32)
class LogEntry(models.Model):
person = models.ForeignKey(Person, related_name='log_entries')
created = models.DateTimeField(default=datetime.now)
}}}
the following function fails with
{{{
#!python
"TypeError: expected string or Unicode object, NoneType found":
}}}
{{{
#!python
def pickle_persons_with_latest_logentry_time():
qs =
Person.objects.all().annotate(latest_logentry_time=Max('log_entries__created'))
pickled = cPickle.dumps(qs)
}}}
Changing to using the pickle module instead of cPickle changes the
exception to:
{{{
#!python
PicklingError: Can't pickle <built-in method now of type object at
0x7f3e064bb020>: it's not found as __main__.now
}}}
so this looks related to #13328
Tested with Django 1.3.1 on python 2.4.3 (Centos 5), Django 1.3.1 on
python 2.7 (Fedora 14), and latest Django-SVN with python 2.7, and got the
same result on all of them.
--
Ticket URL: <https://code.djangoproject.com/ticket/17917>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
fails
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
queryset,pickle,aggregation | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by aaugustin):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/17917#comment:1>
Django
fails
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
queryset,pickle,aggregation | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by aaugustin):
Indeed, I could reproduce the problem. Pickling works without the
annotation and crashes with the annotation.
{{{
>>> import pickle
>>> import cPickle
>>> from django.db.models import Max
>>> from test_app.models import Person
>>> qs = Person.objects.all()
>>> pickled = cPickle.dumps(qs)
>>> pickled = pickle.dumps(qs)
>>> qs =
Person.objects.all().annotate(latest_logentry_time=Max('log_entries__created'))
>>> pickled = cPickle.dumps(qs)
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found
>>> pickled = pickle.dumps(qs)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1374, in dumps
Pickler(file, protocol).dump(obj)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 224, in dump
self.save(obj)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 401, in save_reduce
save(args)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 562, in save_tuple
save(element)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 748, in save_global
(obj, module, name))
PicklingError: Can't pickle <built-in method now of type object at
0x102ab8420>: it's not found as __main__.now
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/17917#comment:2>
Django
fails
Reporter: jensadne@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
queryset,pickle,aggregation |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* has_patch: 0 => 1
Comment:
Fixed by f403653cf146384946e5c879ad2a351768ebc226 in Django 1.6.
[https://github.com/django/django/pull/5410 PR] to add a unit test.
Annotating across a relation isn't required to trigger the exception.
--
Ticket URL: <https://code.djangoproject.com/ticket/17917#comment:4>
Django
fails
Reporter: jensadne@… | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
queryset,pickle,aggregation |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"ca43ee9d0f4682c989e6b88bbc97ddb706ecff0a" ca43ee9]:
{{{
#!CommitTicketReference repository=""
revision="ca43ee9d0f4682c989e6b88bbc97ddb706ecff0a"
Refs #17917 -- Added a test for pickling annotations on fields with
callable defaults.
Fixed in f403653cf146384946e5c879ad2a351768ebc226.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/17917#comment:5>
Django
fails
Reporter: jensadne@… | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Keywords: | Triage Stage: Accepted
queryset,pickle,aggregation |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => fixed
--
Ticket URL: <https://code.djangoproject.com/ticket/17917#comment:6>