[Django] #36288: Regression in values_list() with duplicated field names

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Release
| blocker
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
In Django 5.2, a call to values_list("field", "field") will return tuples
with a single member, as duplicated field names seem to be filtered out
somewhere.

I suspect a regression caused by #28900.
--
Ticket URL: <https://code.djangoproject.com/ticket/36288>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------

Reporter: Claude Paroz | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* cc: Simon Charette (added)
* stage: Unreviewed => Accepted

Comment:

Thank you Claude for the report! I have reproduced with the following:
{{{
>>> import django
>>> django.VERSION
(6, 0, 0, 'alpha', 0)
>>> [i for i in User.objects.all().values_list("username", "username")]
[('admin',)]
}}}

Versus:
{{{
>>> import django
>>> django.VERSION
(5, 1, 7, 'final', 0)
>>> from django.contrib.auth.models import User
>>> [i for i in User.objects.all().values_list("username", "username")]
[('admin', 'admin')]
}}}

Now, I'm not sure this is a bug? I mean there is certainly a change in
behavior, but to me the 5.1 result feels "buggy", so in a way this "was
fixed" in 5.2.
On the other hand, the DB query does duplicate the columns so perhaps this
*is* a regression:
{{{#!sql
psql (16.8 (Ubuntu 16.8-0ubuntu0.24.04.1))
Type "help" for help.

djangotest=# select username, username from auth_user;
username | username
----------+----------
admin | admin
(1 row)
}}}

Claude, could you indulge me and share in the ticket the use case for
getting the repeated fields in the `values_list` result? Accepting in the
meantime.
--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:1>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------

Reporter: Claude Paroz | Owner: Simon
| Charette
Type: Bug | Status: assigned

Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* owner: (none) => Simon Charette
* status: new => assigned

Comment:

This is effectively a regression caused by #28900
(55bacb7a2a7b65c0708d0b2f28e1f9c83c76509e) as `sql.Query.selected` is now
a `dict[alias, Expression]` and since `values_list` defaults to using the
provided strings as field names they get considered to be a single one.

We'll most likely have to generate an alias for the existing members with
colliding names.
--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:2>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:3>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Claude Paroz):

Thanks for the quick reaction! FWIW, the use case that revealed this was a
QuerySet to produce tuples for form choices where the value and the
verbose parts were identical.
--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:4>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:5>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Owner: Simon
| Charette

Type: Bug | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"21f8be76d43aa1ee5ae41c1e0a428cfea1f231c1" 21f8be7]:
{{{#!CommitTicketReference repository=""
revision="21f8be76d43aa1ee5ae41c1e0a428cfea1f231c1"
Fixed #36288 -- Addressed improper handling of duplicates in
values_list().

Now that selected aliases are stored in sql.Query.selected: dict[str, Any]
the values_list() method must ensures that duplicate field name references
are
assigned unique aliases.

Refs #28900.

Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a.

Thanks Claude for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:6>

[Django]

#36288: Regression in values_list() with duplicated field names
-------------------------------------+-------------------------------------
Reporter: Claude Paroz | Owner: Simon
| Charette
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"b97af5e696b235876c1a1c8d8a704580f2aeabc2" b97af5e]:
{{{#!CommitTicketReference repository=""
revision="b97af5e696b235876c1a1c8d8a704580f2aeabc2"
[5.2.x] Fixed #36288 -- Addressed improper handling of duplicates in

values_list().

Now that selected aliases are stored in sql.Query.selected: dict[str, Any]
the values_list() method must ensures that duplicate field name references
are
assigned unique aliases.

Refs #28900.

Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a.

Thanks Claude for the report.

Backport of 21f8be76d43aa1ee5ae41c1e0a428cfea1f231c1 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36288#comment:7>