[Django] #26676: prefetch_related incorrectly returns list instead of queryset when using Prefetch with to_attr
Django
Prefetch with to_attr
-------------------------------------+-------------------------------------
Reporter: Ursidours | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: master
(models, ORM) | Keywords: Prefetch
Severity: Normal | prefetch_related queryset
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
When using prefetch_related with a Prefetch whose to_attr is set, the
related object returns a list instead of a queryset, making chaining
queries impossible. This bug affects both version 1.9 and 1.10, but to a
different extent.
This minimal example illustrates the issue, the test fails with
{{{AttributeError: 'list' object has no attribute 'filter'}}}
'''model''''
{{{
from django.db import models
class Book(models.Model):
name = models.CharField(max_length=12)
class Page(models.Model):
number = models.SmallIntegerField()
text = models.TextField()
book = models.ForeignKey(Book)
}}}
'''tests.py'''
{{{
from django.test import TestCase
from alpha_test.models import Book, Page
from django.db.models import Q, Prefetch
class SimpleCase(TestCase):
def setUp(self):
book = Book.objects.create(name="Django Textbook")
page1 = Page.objects.create(number=1, text="lorem", book=book)
page2 = Page.objects.create(number=2, text="ipsum", book=book)
def test_prefech_related_with_Prefetch(self):
queryset = Page.objects.filter(number=1)
prefetch_no_attr = Prefetch('page_set', queryset=queryset)
prefetch_with_attr = Prefetch('page_set', queryset=queryset,
to_attr='first_page')
# Selecting the book with the Prefetch without to_attr
# it is possible to chain queries on the page_set
book = Book.objects.prefetch_related(prefetch_no_attr).first()
first_page = book.page_set.filter(number=1) # works fine
# Now with Prefetch with to_attr set to some value,
# the same query on page_set fails for 1.10 (but works for 1.9).
book = Book.objects.prefetch_related(prefetch_with_attr).first()
first_page = book.page_set.filter(number=1)
# However, querying on the to_attr fails for both 1.9 and 1.10
first_page = book.first_page.filter(text="lorem")
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26676>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
cache
-------------------------------------+-------------------------------------
Reporter: Ursidours | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: Prefetch | Triage Stage: Accepted
prefetch_related queryset |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* severity: Normal => Release blocker
* needs_better_patch: => 0
* needs_tests: => 0
* version: master => 1.10
* needs_docs: => 0
* stage: Unreviewed => Accepted
Comment:
Hi Ursidours,
Thanks for your very detailed report.
As documented, when using `to_attr` the
[https://docs.djangoproject.com/en/1.9/ref/models/querysets/#django.db.models.Prefetch
prefetched result is stored in a list]. If you'd like to request this to
be changed I suggest you open a new ticket for this purpose.
Now, using `to_attr` shouldn't prefetch and store results for the
referenced relation (unless the referenced relation equals `to_attr`). It
should be left untouched and that's a regression in 1.10 introduced by
bdbe50a491ca41e7d4ebace47bfe8abe50a58211.
Based on your model definition a simplified test case could look like:
{{{#!python
first_pages = Page.objects.filter(number=1)
books = Book.objects.prefetch_related(
Prefetch('page_set', queryset=first_pages, to_attr='first_pages')
)
book = book.first()
assert list(book.page_set.filter(number=1)) == book.first_pages
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26676#comment:1>
Django
cache
Reporter: Ursidours | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: Prefetch | Triage Stage: Accepted
prefetch_related queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/6672 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/26676#comment:2>
Django
cache
Reporter: Ursidours | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Release blocker | Resolution:
prefetch_related queryset | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/26676#comment:3>
Django
cache
Reporter: Ursidours | Owner: nobody
Component: Database layer | Version: 1.10
(models, ORM) |
Keywords: Prefetch | Triage Stage: Ready for
prefetch_related queryset | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"53a5fb3cc0137bebeebc0d4d321dbfe20397b065" 53a5fb3c]:
{{{
#!CommitTicketReference repository=""
revision="53a5fb3cc0137bebeebc0d4d321dbfe20397b065"
Fixed #26676 -- Prevented prefetching to_attr from caching its result in
through attr.
Thanks Ursidours for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26676#comment:4>
Django
cache
Reporter: Ursidours | Owner: nobody
Component: Database layer | Version: 1.10
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: Prefetch | Triage Stage: Ready for
prefetch_related queryset | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette <charette.s@…>):
In [changeset:"58f0d40b6d98bf9ad2861cf742f7b2a327cc8068" 58f0d40b]:
{{{
#!CommitTicketReference repository=""
revision="58f0d40b6d98bf9ad2861cf742f7b2a327cc8068"
[1.10.x] Fixed #26676 -- Prevented prefetching to_attr from caching its
result in through attr.
Thanks Ursidours for the report.
Backport of 53a5fb3cc0137bebeebc0d4d321dbfe20397b065 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26676#comment:5>