[Django] #36326: Add support for CompositePrimaryKey in QuerySet.raw()

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: dev | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Currently, a check in `RawModelIterable` fails for models with a
`CompositePrimaryKey`.

For example, this model:

{{{#!python
class AuthorEvent(models.Model):
timestamp = models.DateTimeField(auto_now_add=True)
author = models.ForeignKey(Author, on_delete=models.CASCADE)
pk = models.CompositePrimaryKey(
"author",
"timestamp",
)
}}}

fails like:

{{{
In [2]: list(AuthorEvent.objects.raw("SELECT * FROM example_authorevent"))
---------------------------------------------------------------------------
FieldDoesNotExist Traceback (most recent call
last)
Cell In[2], line 1
----> 1 list(AuthorEvent.objects.raw("SELECT * FROM example_authorevent"))

File ..../django/db/models/query.py:2133, in RawQuerySet.__iter__(self)
2132 def __iter__(self):
-> 2133 self._fetch_all()
2134 return iter(self._result_cache)

File ..../django/db/models/query.py:2120, in RawQuerySet._fetch_all(self)
2118 def _fetch_all(self):
2119 if self._result_cache is None:
-> 2120 self._result_cache = list(self.iterator())
2121 if self._prefetch_related_lookups and not self._prefetch_done:
2122 self._prefetch_related_objects()

File ..../django/db/models/query.py:2147, in RawQuerySet.iterator(self)
2146 def iterator(self):
-> 2147 yield from RawModelIterable(self)

File ..../django/db/models/query.py:169, in
RawModelIterable.__iter__(self)
167 model_cls = self.queryset.model
168 if model_cls._meta.pk.attname not in model_init_names:
--> 169 raise exceptions.FieldDoesNotExist(
170 "Raw query must include the primary key"
171 )
172 fields = [self.queryset.model_fields.get(c) for c in
self.queryset.columns]
173 cols = [f.get_col(f.model._meta.db_table) if f else None for f in
fields]

FieldDoesNotExist: Raw query must include the primary key
}}}

To add support, we should add a test, update this check to look for all
columns in the PK, and fix anything else as necessary.
--
Ticket URL: <https://code.djangoproject.com/ticket/36326>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------

Reporter: Adam Johnson | Owner: (none)
Type: New feature | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* cc: Csirmaz Bendegúz, Simon Charette (added)
* stage: Unreviewed => Accepted
* type: Bug => New feature

Comment:

Thank you for the report!
I've marked this as a new feature to add support but I can be persuaded to
have this as a release blocking bug

Replicated:
{{{#!diff
--- a/tests/composite_pk/tests.py
+++ b/tests/composite_pk/tests.py
@@ -158,6 +158,10 @@ class CompositePKTests(TestCase):
users = User.objects.values_list("pk").order_by("pk")
self.assertNotIn('AS "pk"', str(users.query))

+ def test_raw(self):
+ users = User.objects.raw("SELECT * FROM composite_pk_user")
+ self.assertEqual(len(users), 1)
+
def test_only(self):
users = User.objects.only("pk")
self.assertSequenceEqual(users, (self.user,))
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:1>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: (none)
Type: New feature | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Csirmaz Bendegúz):

* easy: 0 => 1

Comment:

thanks for the cc! I see, we have overlooked raw queries. I wasn't aware
of this check there

agreed it's not release blocking, but it does feel like a bug

it should be an easy fix, marking it as "easy pickings" in case a newcomer
wants to pick it up
--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:2>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:

| JaeHyuckSa
Type: New feature | Status: assigned

Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by JaeHyuckSa):

* owner: (none) => JaeHyuckSa
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:3>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:
| JaeHyuckSa
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Johnson):

Yeah, I wouldn't make this a release blocker, just another "extra support"
feature ticket :)
--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:4>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:
| JaeHyuckSa
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by JaeHyuckSa):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:5>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:
| JaeHyuckSa
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:6>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:
| JaeHyuckSa

Type: New feature | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"1831f7733d3ef03d1ca7fac3e8d9f4c5e3e3375e" 1831f77]:
{{{#!CommitTicketReference repository=""
revision="1831f7733d3ef03d1ca7fac3e8d9f4c5e3e3375e"
Fixed #36326 -- Added CompositePrimaryKey support in QuerySet.raw().

Signed-off-by: SaJH <wogur...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:7>

[Django]

#36326: Add support for CompositePrimaryKey in QuerySet.raw()
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner:
| JaeHyuckSa
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"045110ff3089aefd9c3e65c707df465bacfed986" 045110ff]:
{{{#!CommitTicketReference repository=""
revision="045110ff3089aefd9c3e65c707df465bacfed986"
Refs #36326 -- Fixed QuerySet.raw() crash on models with
CompositePrimaryKey on Oracle.

Virtual CompositePrimaryKey fields should be ignored.

Regression in 1831f7733d3ef03d1ca7fac3e8d9f4c5e3e3375e.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36326#comment:8>