[Django] #34570: QuerySet.defer() raises an AttributeError when the field is ManyToManyField or GenericForeignKey
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco | Owner: nobody
Martínez |
Type: Bug | Status: new
Component: Database | Version: 4.2
layer (models, ORM) | Keywords: contenttypes,
Severity: Normal | defer, manytomanyfield,
Triage Stage: | genericforeignkey
Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
== In Version 4.2.1
Hello, when I try to defer a field that is a ''ManyToManyField'' or a
''GenericForeignKey'' it raises an AttributteError.
=== Having these three models:
{{{
from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey,
ContentType
class ModelA(models.Model):
name = models.CharField(max_length=100)
class ModelB(models.Model):
name = models.CharField(max_length=100)
model_a = models.ManyToManyField('ModelA')
class ModelC(models.Model):
name = models.CharField(max_length=100)
content_type = models.ForeignKey(ContentType, null=True,
on_delete=models.CASCADE)
object_id = models.PositiveIntegerField(null=True)
generic_model = GenericForeignKey('content_type', 'object_id')
}}}
== Traceback from a ManyToManyField:
{{{
>>> ModelB.objects.defer('model_a')
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 374, in __repr__
data = list(self[: REPR_OUTPUT_SIZE + 1])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 398, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 1881, in _fetch_all
self._result_cache = list(self._iterable_class(self))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 91, in __iter__
results = compiler.execute_sql(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
sql, params = self.as_sql()
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 734, in as_sql
extra_select, order_by, group_by = self.pre_sql_setup(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 84, in pre_sql_setup
self.setup_query(with_col_aliases=with_col_aliases)
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 73, in setup_query
self.select, self.klass_info, self.annotation_col_map =
self.get_select(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 256, in get_select
select_mask = self.query.get_select_mask()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/query.py", line 772, in get_select_mask
return self._get_defer_select_mask(opts, mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/query.py", line 728, in
_get_defer_select_mask
field = opts.get_field(field_name).field
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ManyToManyField' object has no attribute 'field'
}}}
== Traceback from defering a GenericForeignKey:
{{{
>>> ModelC.objects.defer('generic_model')
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 374, in __repr__
data = list(self[: REPR_OUTPUT_SIZE + 1])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 398, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 1881, in _fetch_all
self._result_cache = list(self._iterable_class(self))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/query.py", line 91, in __iter__
results = compiler.execute_sql(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
sql, params = self.as_sql()
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 734, in as_sql
extra_select, order_by, group_by = self.pre_sql_setup(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 84, in pre_sql_setup
self.setup_query(with_col_aliases=with_col_aliases)
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 73, in setup_query
self.select, self.klass_info, self.annotation_col_map =
self.get_select(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/compiler.py", line 256, in get_select
select_mask = self.query.get_select_mask()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/query.py", line 772, in get_select_mask
return self._get_defer_select_mask(opts, mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/db/models/sql/query.py", line 728, in
_get_defer_select_mask
field = opts.get_field(field_name).field
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GenericForeignKey' object has no attribute 'field'
}}}
== PD:
In version 4.1.9 it is working without any error, but in versions 4.2.0
and 4.2.1 raises the ''AttributeError''
--
Ticket URL: <https://code.djangoproject.com/ticket/34570>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: contenttypes, | Triage Stage: Accepted
defer, manytomanyfield, |
genericforeignkey |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Simon Charette (added)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
Thanks for the report!
Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.
Reproduced at 0ec60661e61b153e6bcec64649b1b7f524eb3e18.
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:1>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: contenttypes, | Triage Stage: Accepted
defer, manytomanyfield, |
genericforeignkey |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Simon Charette
* status: new => assigned
Comment:
> In version 4.1.9 it is working without any error, but in versions 4.2.0
and 4.2.1 raises the AttributeError
Could you elaborate on that? Many-to-many fields and generic foreign keys
are not deferrable (they are virtual and do not map to columns) so the
previous code was simply swallowing invalid values passed to `defer`.
I guess that raising a more detailed exception would still require a
deprecation period and that it would be better to perform this validation
at `defer`/`only` instead of compilation time anyway so our only option
here is to silence cases where `opts.get_field(field_name)` returns
something that doesn't have a `.field` property.
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:2>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: contenttypes, | Triage Stage: Accepted
defer, manytomanyfield, |
genericforeignkey |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:3>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
defer, manytomanyfield, | checkin
genericforeignkey |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:4>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Component: Database layer | Version: 4.2
(models, ORM) |
Keywords: contenttypes, | Triage Stage: Ready for
defer, manytomanyfield, | checkin
genericforeignkey |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"99e5dff737cd20b12d060e4794e097063b61ec40" 99e5dff7]:
{{{
#!CommitTicketReference repository=""
revision="99e5dff737cd20b12d060e4794e097063b61ec40"
Fixed #34570 -- Silenced noop deferral of many-to-many and GFK.
While deferring many-to-many and GFK has no effect, the previous
implementation of QuerySet.defer() ignore them instead of crashing.
Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.
Thanks Paco Martínez for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:5>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: contenttypes, | Triage Stage: Ready for
defer, manytomanyfield, | checkin
genericforeignkey |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"201d29b3719ef15637648be7bd947ef90a66ab55" 201d29b]:
{{{
#!CommitTicketReference repository=""
revision="201d29b3719ef15637648be7bd947ef90a66ab55"
[4.2.x] Fixed #34570 -- Silenced noop deferral of many-to-many and GFK.
While deferring many-to-many and GFK has no effect, the previous
implementation of QuerySet.defer() ignore them instead of crashing.
Regression in b3db6c8dcb5145f7d45eff517bcd96460475c879.
Thanks Paco Martínez for the report.
Backport of 99e5dff737cd20b12d060e4794e097063b61ec40 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:6>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: contenttypes, | Triage Stage: Ready for
defer, manytomanyfield, | checkin
genericforeignkey |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Paco Martínez):
Replying to [comment:2 Simon Charette]:
> > In version 4.1.9 it is working without any error, but in versions
4.2.0 and 4.2.1 raises the AttributeError
>
> Could you elaborate on that? Many-to-many fields and generic foreign
keys are not deferrable (they are virtual and do not map to columns) so
the previous code was simply swallowing invalid values passed to `defer`.
>
> I guess that raising a more detailed exception would still require a
deprecation period and that it would be better to perform this validation
at `defer`/`only` instead of compilation time anyway so our only option
here is to silence cases where `opts.get_field(field_name)` returns
something that doesn't have a `.field` property.
Well, the more I read about `defer()` and what it was for, the more I
wondered that we were deferring ''ManyToMany'' and ''GenericForeignKey''
fields. Thanks for confirming that it was useless and that it just
swallowed it without taking any action.
Thank you very much to both of you for your answers and for your quickness
in solving it.
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:7>
Django
ManyToManyField or GenericForeignKey
-------------------------------------+-------------------------------------
Reporter: Paco Martínez | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: contenttypes, | Triage Stage: Ready for
defer, manytomanyfield, | checkin
genericforeignkey |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"d9e701879612312391c0dca5c158b79a1dabcf18" d9e7018]:
{{{
#!CommitTicketReference repository=""
revision="d9e701879612312391c0dca5c158b79a1dabcf18"
Refs #34570 -- Added extra tests for QuerySet.only() noops.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34570#comment:8>