[Django] #36373: select_related() doesn't work when targeting composite primary keys

Django

unread,

May 7, 2025, 11:24:37 AM5/7/25

to django-...@googlegroups.com

#36373: select_related() doesn't work when targeting composite primary keys
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
The [https://docs.djangoproject.com/en/5.2/topics/composite-primary-key
/#composite-primary-keys-and-relations suggested workaround] until #35956
implements foreign keys to models with composite primary keys is to use
`ForeignObject`, but it causes failures in `select_related()`. (Of note:
`prefetch_related()` works fine.)

{{{#!diff
diff --git a/tests/composite_pk/tests.py b/tests/composite_pk/tests.py
index 5dea23c9f2..068030bd18 100644
--- a/tests/composite_pk/tests.py
+++ b/tests/composite_pk/tests.py
@@ -184,6 +184,11 @@ class CompositePKTests(TestCase):
with self.assertNumQueries(1):
self.assertEqual(user.email, self.user.email)

+ def test_select_related(self):
+ with self.assertNumQueries(1):
+ for comment in Comment.objects.select_related():
+ comment.user
+
def test_model_forms(self):
fields = ["tenant", "id", "user_id", "text", "integer"]
self.assertEqual(list(CommentForm.base_fields), fields)
}}}
{{{
======================================================================
ERROR: test_select_related
(composite_pk.tests.CompositePKTests.test_select_related)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/.../django/tests/composite_pk/tests.py", line 189, in
test_select_related
for comment in Comment.objects.select_related():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Users/.../django/django/db/models/query.py", line 403, in
__iter__
self._fetch_all()
~~~~~~~~~~~~~~~^^
File "/Users/.../django/django/db/models/query.py", line 1966, in
_fetch_all
self._result_cache = list(self._iterable_class(self))
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/.../django/django/db/models/query.py", line 107, in
__iter__
related_populators = get_related_populators(klass_info, select, db,
fetch_mode)
File "/Users/.../django/django/db/models/query.py", line 2742, in
get_related_populators
rel_cls = RelatedPopulator(rel_klass_info, select, db, fetch_mode)
File "/Users/.../django/django/db/models/query.py", line 2710, in
__init__
self.pk_idx = self.init_list.index(self.model_cls._meta.pk.attname)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: 'pk' is not in list
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36373>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

May 7, 2025, 12:01:12 PM5/7/25

to django-...@googlegroups.com

#36373: select_related() doesn't work when targeting composite primary keys
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thanks for the report, it's worth pointing out that the issue is not
specific to calling `select_related` without a subset of relationship the
following test fails as well and is a more common usage of
`select_related`

{{{#!diff
diff --git a/tests/composite_pk/tests.py b/tests/composite_pk/tests.py

index 5dea23c9f2..6983e13945 100644
--- a/tests/composite_pk/tests.py
+++ b/tests/composite_pk/tests.py
@@ -184,6 +184,11 @@ def test_only(self):

with self.assertNumQueries(1):
self.assertEqual(user.email, self.user.email)

+ def test_select_related(self):
+ with self.assertNumQueries(1):

+ for comment in Comment.objects.select_related("user"):

+ comment.user
+
def test_model_forms(self):
fields = ["tenant", "id", "user_id", "text", "integer"]
self.assertEqual(list(CommentForm.base_fields), fields)
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/36373#comment:1>

Django

unread,

May 7, 2025, 12:17:12 PM5/7/25

to django-...@googlegroups.com

#36373: select_related() doesn't work when targeting composite primary keys
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

Looks like something like the following should do

{{{#!diff
diff --git a/django/db/models/query.py b/django/db/models/query.py
index 4f4aad91ef..0472e99144 100644
--- a/django/db/models/query.py
+++ b/django/db/models/query.py
@@ -2680,7 +2680,12 @@ def __init__(self, klass_info, select, db):
)

self.model_cls = klass_info["model"]
- self.pk_idx =
self.init_list.index(self.model_cls._meta.pk.attname)
+ pk_fields = self.model_cls._meta.pk_fields
+ pk_idx = self.init_list.index(pk_fields[0].attname)
+ if (pk_fields_len := len(pk_fields)) > 1:
+ self.pk_idx = slice(pk_idx, pk_fields_len)
+ else:
+ self.pk_idx = pk_idx
self.related_populators = get_related_populators(klass_info,
select, self.db)
self.local_setter = klass_info["local_setter"]
self.remote_setter = klass_info["remote_setter"]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36373#comment:2>

unread,

May 12, 2025, 7:33:18 AM5/12/25

to django-...@googlegroups.com

#36373: select_related() doesn't work when targeting composite primary keys
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Simon
| Charette

Type: Bug | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"8be0c0d6901669661fca578f474cd51cd284d35a" 8be0c0d]:
{{{#!CommitTicketReference repository=""
revision="8be0c0d6901669661fca578f474cd51cd284d35a"
Fixed #36373 -- Fixed select_related() crash on foreign object for a
composite pk.

Thanks Jacob Walls for the report and Sarah for the in-depth review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36373#comment:8>

Django

unread,

May 12, 2025, 7:35:29 AM5/12/25

to django-...@googlegroups.com

#36373: select_related() doesn't work when targeting composite primary keys
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"e23dd72880bf9975e928ef10ca4fdfa897665026" e23dd728]:
{{{#!CommitTicketReference repository=""
revision="e23dd72880bf9975e928ef10ca4fdfa897665026"
[5.2.x] Fixed #36373 -- Fixed select_related() crash on foreign object for

a composite pk.

Thanks Jacob Walls for the report and Sarah for the in-depth review.

Backport of 8be0c0d6901669661fca578f474cd51cd284d35a from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36373#comment:9>

Reply all

Reply to author

Forward