[Django] #24831: Pickling querysets with prefetch_related breaks

8 views
Skip to first unread message

Django

unread,
May 20, 2015, 12:37:33 PM5/20/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
----------------------------------------------+----------------------------
Reporter: lsemel | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.8
Severity: Normal | Keywords:
| prefetch_related
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+----------------------------
In Django 1.8.1, pickling certain querysets that use prefetch_related
breaks, with an error message that pickle can't find the through model,
with an error like

{{{
PicklingError: Can't pickle <class 'bug.models.Category_books'>: it's
not found as bug.models.Category_books
}}}

With these models

{{{

from django.db import models

class Book(models.Model):
title = models.CharField(max_length=100)

class Author(models.Model):
name = models.CharField(max_length=100)
books = models.ManyToManyField(Book, related_name='authors')

class Category(models.Model):
title = models.CharField(max_length=100)
books = models.ManyToManyField(Book, related_name='categories')

}}}

and these tests

{{{

from django.test import TestCase
import pickle
from .models import Book, Author, Category

class PickleTestCase(TestCase):
def setUp(self):
author1 = Author.objects.create(name='Joe Schmoe')
author2 = Author.objects.create(name='Jane Schmoe')
book1 = Book.objects.create(title='Whatever')
book1.authors.add(author1)
book1.authors.add(author2)
book2 = Book.objects.create(title='Another')
book2.authors.add(author1)
book2.authors.add(author2)
cat1 = Category.objects.create(title='Category 1')
cat1.books.add(book1)
cat1.books.add(book2)
cat2 = Category.objects.create(title='Category 2')
cat2.books.add(book1)
cat2.books.add(book2)

def tearDown(self):
Author.objects.all().delete()
Book.objects.all().delete()
Category.objects.all().delete()

def test_pickle(self):
qs = Category.objects.prefetch_related('books__authors').all()
pickle.dumps(qs)

def test_pickle2(self):
qs = Author.objects.prefetch_related('books__categories').all()
pickle.dumps(qs)

def test_pickle3(self):
qs = Book.objects.prefetch_related('categories__books').all()
pickle.dumps(qs)


}}}

The second and third tests will fail with the following stack trace

{{{


.EE
======================================================================
ERROR: test_pickle2 (bug.tests.PickleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/lsemel/www/djangobug/bug/tests.py", line 33, in
test_pickle2
pickle.dumps(qs)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1374, in dumps
Pickler(file, protocol).dump(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 224, in dump
self.save(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 600, in save_list
self._batch_appends(iter(obj))
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 615, in _batch_appends
save(x)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 600, in save_list
self._batch_appends(iter(obj))
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 615, in _batch_appends
save(x)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 300, in save
self.save_global(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 748, in save_global
(obj, module, name))
PicklingError: Can't pickle <class 'bug.models.Category_books'>:
it's not found as bug.models.Category_books

======================================================================
ERROR: test_pickle3 (bug.tests.PickleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/lsemel/www/djangobug/bug/tests.py", line 37, in
test_pickle3
pickle.dumps(qs)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1374, in dumps
Pickler(file, protocol).dump(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 224, in dump
self.save(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 600, in save_list
self._batch_appends(iter(obj))
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 615, in _batch_appends
save(x)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 600, in save_list
self._batch_appends(iter(obj))
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 615, in _batch_appends
save(x)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 331, in save
self.save_reduce(obj=obj, *rv)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 419, in save_reduce
save(state)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 286, in save
f(self, obj) # Call unbound method with explicit self
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 649, in save_dict
self._batch_setitems(obj.iteritems())
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 663, in _batch_setitems
save(v)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 300, in save
self.save_global(obj)
File
"/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 748, in save_global
(obj, module, name))
PicklingError: Can't pickle <class 'bug.models.Category_books'>:
it's not found as bug.models.Category_books

----------------------------------------------------------------------
Ran 3 tests in 0.075s


}}}

If the deletions in tearDown() are removed, all the tests pass.

--
Ticket URL: <https://code.djangoproject.com/ticket/24831>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
May 20, 2015, 3:57:06 PM5/20/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------

Reporter: lsemel | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by timgraham):

* needs_better_patch: => 0
* needs_docs: => 0
* severity: Normal => Release blocker
* needs_tests: => 0
* stage: Unreviewed => Accepted


Comment:

Regression in 1.8 bisected to f233bf47dde1d481108142c8d6b4bb3b3d8c6d08.

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:1>

Django

unread,
May 28, 2015, 4:52:21 AM5/28/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------

Reporter: lsemel | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by coldmind):

* Attachment "ticket_24831.patch" added.

test to reproduce the issue

Django

unread,
May 28, 2015, 4:53:37 AM5/28/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------

Reporter: lsemel | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by coldmind):

* cc: me@… (added)


Comment:

Problem is near
https://github.com/django/django/blob/300e8baf9429dbf1c5dff478558d793638dc2f54/django/db/models/deletion.py#L226-L231

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:2>

Django

unread,
May 28, 2015, 11:27:52 AM5/28/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------

Reporter: lsemel | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by timgraham):

The cause could be similar to #24381 (pickling of some cached_properties
that the delete code initializes).

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:3>

Django

unread,
May 29, 2015, 7:52:10 AM5/29/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------
Reporter: lsemel | Owner: coldmind
Type: Bug | Status: assigned

Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by coldmind):

* owner: nobody => coldmind
* status: new => assigned
* has_patch: 0 => 1


Comment:

https://github.com/django/django/pull/4720

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:4>

Django

unread,
Jun 1, 2015, 3:21:29 PM6/1/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------
Reporter: lsemel | Owner: coldmind
Type: Bug | Status: assigned
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by timgraham):

* needs_better_patch: 0 => 1


--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:5>

Django

unread,
Jun 2, 2015, 9:08:25 AM6/2/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------
Reporter: lsemel | Owner: coldmind
Type: Bug | Status: assigned
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by coldmind):

* needs_better_patch: 1 => 0


--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:6>

Django

unread,
Jun 2, 2015, 10:06:30 AM6/2/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------
Reporter: lsemel | Owner: coldmind
Type: Bug | Status: closed

Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution: fixed

Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham <timograham@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"2913d6b77d8b2082dc79df5503b7dd3ddd05fcc1" 2913d6b7]:
{{{
#!CommitTicketReference repository=""
revision="2913d6b77d8b2082dc79df5503b7dd3ddd05fcc1"
Fixed #24831 -- Fixed pickling queryset with prefetch_related() after
deleting objects.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:7>

Django

unread,
Jun 2, 2015, 10:08:03 AM6/2/15
to django-...@googlegroups.com
#24831: Pickling querysets with prefetch_related breaks
-------------------------------------+-------------------------------------
Reporter: lsemel | Owner: coldmind
Type: Bug | Status: closed
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham <timograham@…>):

In [changeset:"9d83de8ff3b835603fe09e8842529c9810bc1dcd" 9d83de8f]:
{{{
#!CommitTicketReference repository=""
revision="9d83de8ff3b835603fe09e8842529c9810bc1dcd"
[1.8.x] Fixed #24831 -- Fixed pickling queryset with prefetch_related()
after deleting objects.

Backport of 2913d6b77d8b2082dc79df5503b7dd3ddd05fcc1 from master
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/24831#comment:8>

Reply all
Reply to author
Forward
0 new messages