[Django] #30482: prefetch_related_objects evaluates lookup.queryset when checking for invalid prefetch ordering
Django
invalid prefetch ordering
-------------------------------------+-------------------------------------
Reporter: Iain P. | Owner: (none)
Watts |
Type: | Status: assigned
Uncategorized |
Component: Database | Version: 2.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When encountering a prefetch it has already seen and checking whether to
throw the "lookup was already seen with a different queryset" ValueError,
prefetch_related_objects will inadvertently evaluate lookup.queryset. For
example:
{{{
foo = Foo.objects.create()
prefetches = [
'bar',
Prefetch('bar', queryset=Bar.objects.filter(my_field=True))
]
prefetch_related_objects([foo], *prefetches)
}}}
Right before it throws the ValueError, the above code will execute the
full unfiltered query `Bar.objects.filter(my_field=True)`. This may match
a very large number of rows (or a whole table), since it is not restricted
to `Bars` related to the specific `foo`. In our case this led to the query
timing out and the page returning a 504 without the code ever actually
reaching the ValueError.
--
Ticket URL: <https://code.djangoproject.com/ticket/30482>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
invalid prefetch ordering
-------------------------------------+-------------------------------------
| Watts
Type: Uncategorized | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => Iain P. Watts
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:1>
Django
invalid prefetch ordering
-------------------------------------+-------------------------------------
| Watts
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* type: Uncategorized => Bug
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:2>
Django
invalid prefetch ordering
-------------------------------------+-------------------------------------
| Watts
Type: Bug | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:3>
Django
invalid prefetch ordering
-------------------------------------+-------------------------------------
| Watts
Type: Bug | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Iain P. Watts):
https://github.com/django/django/pull/11369
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:4>
Django
prefetch ordering.
-------------------------------------+-------------------------------------
Reporter: Iain P. Watts | Owner: Iain P.
| Watts
Type: Bug | Status: assigned
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 2.2 => master
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:5>
Django
prefetch ordering.
-------------------------------------+-------------------------------------
Reporter: Iain P. Watts | Owner: Iain P.
| Watts
Component: Database layer | Version: master
(models, ORM) |
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"59ffafa1d2f466fa7aacbe74fe2787b2004b7871" 59ffafa1]:
{{{
#!CommitTicketReference repository=""
revision="59ffafa1d2f466fa7aacbe74fe2787b2004b7871"
Fixed #30482 -- Prevented unnecessary evaluation of lookup.queryset in
prefetch_related_objects().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30482#comment:6>