[Django] #32627: Multi-model union fails with .first()

[Django]

#32627: Multi-model union fails with .first()
-------------------------------------+-------------------------------------
Reporter: Iuri de | Owner: nobody
Silvio |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Django 3.2 fails with this query:

{{{#!python
qs1 = Celebrity.objects.all()
qs2 = ReservedName.objects.all()
qs1.union(qs2).values_list('name').first()
}}}

It worked until Django 3.1.8. This commit[1] to be exactly.

[1]
https://github.com/django/django/commit/464a4c0c59277056b5d3c1132ac1b4c6085aee08

This is the broken generated query. In the second query, it fetches from
the first table.

{{{#!sql
SQL
SELECT
"queries_celebrity"."name",
"queries_celebrity"."id" AS "__orderbycol2"
FROM
"queries_celebrity"
UNION
SELECT
"queries_reservedname"."name",
"queries_celebrity"."id" AS "__orderbycol2" -- HERE IS THE PROBLEM
FROM
"queries_reservedname"
ORDER BY
(2) ASC
LIMIT
1
}}}

Before, it was:

{{{#!sql
SQL
SELECT
"queries_celebrity"."name",
"queries_celebrity"."id"
FROM
"queries_celebrity"
UNION
SELECT
"queries_reservedname"."name",
"queries_reservedname"."id"
FROM
"queries_reservedname"
ORDER BY
(2) ASC
LIMIT
1
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/32627>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#32627: Multi-model union fails with .first()
-------------------------------------+-------------------------------------

Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Iuri de Silvio):

* owner: nobody => Iuri de Silvio
* status: new => assigned

Old description:

New description:

Django 3.2 fails with this query:

{{{#!python
qs1 = Celebrity.objects.all()
qs2 = ReservedName.objects.all()
qs1.union(qs2).values_list('name').first()
}}}

It worked until Django 3.1.8. This commit[1] to be exactly.

https://github.com/django/django/commit/464a4c0c59277056b5d3c1132ac1b4c6085aee08

Before, it was:

--

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:1>

[Django]

#32627: Multi-model union fails with .first()
-------------------------------------+-------------------------------------

Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Iuri de Silvio):

I did a hacky patch just to make it work with minimal changes.
https://github.com/django/django/pull/14241/commits/9d49fbfee7053c0a17108fe7a908b8956a6181dc

It was easier for me to understand the fix, but I'll cleanup the code.

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:2>

[Django]

#32627: Multi-model union fails with .first()
-------------------------------------+-------------------------------------

Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |

Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* cc: David Wobrock (added)
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thanks for this report.

[https://github.com/django/django/pull/14241 PR]

Regression in 464a4c0c59277056b5d3c1132ac1b4c6085aee08.
Reproduced at 1c004939d5dd781fc16a8b4fb9696661cf545929.

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:3>

[Django]

#32627: Multi-model union fails with .first()
-------------------------------------+-------------------------------------

Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Iuri de Silvio):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:4>

[Django]

#32627: QuerySet.values()/values_list() crash on combined querysets ordered by
unannotated columns.

-------------------------------------+-------------------------------------
Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:5>

[Django]

#32627: QuerySet.values()/values_list() crash on combined querysets ordered by
unannotated columns.
-------------------------------------+-------------------------------------
Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio

Type: Bug | Status: closed

Component: Database layer | Version: 3.2
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* status: assigned => closed
* resolution: => fixed

Comment:

In [changeset:"9760e262f85ae57df39abe2799eff48a82b14474" 9760e26]:
{{{
#!CommitTicketReference repository=""
revision="9760e262f85ae57df39abe2799eff48a82b14474"
Fixed #32627 -- Fixed QuerySet.values()/values_list() crash on combined

querysets ordered by unannotated columns.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:6>

[Django]

#32627: QuerySet.values()/values_list() crash on combined querysets ordered by
unannotated columns.
-------------------------------------+-------------------------------------
Reporter: Iuri de Silvio | Owner: Iuri de
| Silvio
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"b245845575154905701a71d34a4d0c4d21f2f6b3" b2458455]:
{{{
#!CommitTicketReference repository=""
revision="b245845575154905701a71d34a4d0c4d21f2f6b3"
[3.2.x] Fixed #32627 -- Fixed QuerySet.values()/values_list() crash on

combined querysets ordered by unannotated columns.

Backport of 9760e262f85ae57df39abe2799eff48a82b14474 from main
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/32627#comment:7>