[Django] #30796: Chaining select_related mutates original QuerySet.
Django
-------------------------------------+-------------------------------------
Reporter: Darren | Owner: nobody
Maki |
Type: Bug | Status: new
Component: Database | Version: 2.2
layer (models, ORM) | Keywords: select_related
Severity: Normal | prefetch_related mutate queryset
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When creating a new QuerySet from an existing QuerySet that has had
'select_related' applied, if you apply another 'select_related' to the new
QuerySet it will mutate the original QuerySet to also have the extra
'select_related'.
'''models.py'''
{{{
from django.db import models
class ModelA(models.Model):
pass
class ModelB(models.Model):
pass
class ModelC(models.Model):
model_a = models.ForeignKey('foobar.ModelA', on_delete=models.CASCADE)
model_b = models.ForeignKey('foobar.ModelB', on_delete=models.CASCADE)
}}}
'''test.py'''
{{{
query_1 = ModelC.objects.select_related('model_a')
print('QUERY 1:', str(query_1.query))
query_2 = query_1.select_related('model_b')
print('QUERY 2:', str(query_2.query))
print('QUERY 1:', str(query_1.query))
if str(query_1.query) == str(query_2.query):
print('\n!!! The two queries are the same !!!\n')
}}}
'''output'''
{{{
QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
"foobar_modelc"."model_b_id", "foobar_modela"."id" FROM "foobar_modelc"
INNER JOIN "foobar_modela" ON ("foobar_modelc"."model_a_id" =
"foobar_modela"."id") 139663365718536
QUERY 2: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
"foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
"foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
139663366175376
QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
"foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
"foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
139663365718536
!!! The two queries are the same !!!
}}}
The expectation is that the original QuerySet is not mutated, and the two
queries are different. This behavior also happens with 'prefetch_related'.
Since the QuerySet methods call 'self._clone()', and state that they
return 'a new QuerySet instance', this behavior does not seem correct.
--
Ticket URL: <https://code.djangoproject.com/ticket/30796>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: select_related | Triage Stage:
prefetch_related mutate queryset | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
'''models.py'''
{{{
from django.db import models
class ModelA(models.Model):
pass
class ModelB(models.Model):
pass
'''test.py'''
'''output'''
QUERY 2: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
"foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
"foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
"foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
"foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
!!! The two queries are the same !!!
}}}
The expectation is that the original QuerySet is not mutated, and the two
queries are different. This behavior also happens with 'prefetch_related'.
Since the QuerySet methods call 'self._clone()', and state that they
return 'a new QuerySet instance', this behavior does not seem correct.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
prefetch_related mutate queryset |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
This seems to have been happening forever.
`sql.Query.select_related` is made a `dict` on `.add_select_related` but
never copied on `.clone`.
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:2>
Django
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => assigned
* owner: nobody => Simon Charette
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:3>
Django
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
Fixed in https://github.com/django/django/pull/11810
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:4>
Django
-------------------------------------+-------------------------------------
| Charette
Component: Database layer | Version: 2.2
(models, ORM) |
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"37f8f293775d0b672da8ae369d9a4e17f1db7851" 37f8f293]:
{{{
#!CommitTicketReference repository=""
revision="37f8f293775d0b672da8ae369d9a4e17f1db7851"
Fixed #30796 -- Prevented select_related() from mutating a queryset on
chaining.
Thanks Darren Maki for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:5>
Django
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"6b7bd079a6fc4e84553262b65074dbe0af456b03" 6b7bd079]:
{{{
#!CommitTicketReference repository=""
revision="6b7bd079a6fc4e84553262b65074dbe0af456b03"
[3.0.x] Fixed #30796 -- Prevented select_related() from mutating a
queryset on chaining.
Thanks Darren Maki for the report.
Backport of 37f8f293775d0b672da8ae369d9a4e17f1db7851 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:6>
Django
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Alexander Klimenko):
Are there any plans of releasing this fix?
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:7>
Django
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: select_related | Triage Stage: Accepted
prefetch_related mutate queryset |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
It's already released in Django 3.0 and won't be backported to 2.2 LTS
since this bug has been around for a few years before getting identified
and thus isn't a recent regression.
--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:8>