[Django] #36606: Optimize QuerySet.values_list(flat=True) with no fields

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Type:
| Cleanup/optimization
Status: new | Component: Database
| layer (models, ORM)
Version: dev | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Currently, `QuerySet.values_list()` ensures that no more than 1 field is
set
([https://github.com/django/django/blob/41bc48ac1ed1d515977ebe965993b1ef83eafd02/django/db/models/query.py#L1417-L1421
source]):

{{{#!python
if flat and len(fields) > 1:
raise TypeError(
"'flat' is not valid when values_list is called with more
than one "
"field."
)
}}}

However, it also allows the case where *no* fields are declared, for which
all fields are fetched, only to throw away all but the first one
([https://github.com/django/django/blob/41bc48ac1ed1d515977ebe965993b1ef83eafd02/django/db/models/query.py#L266-L278
source]):

{{{#!python
class FlatValuesListIterable(BaseIterable):
"""
Iterable returned by QuerySet.values_list(flat=True) that yields
single
values.
"""

def __iter__(self):
queryset = self.queryset
compiler = queryset.query.get_compiler(queryset.db)
for row in compiler.results_iter(
chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
):
yield row[0]
}}}

I think we can optimize this case to select only the first field in the
model instead, maintaining semantics while avoiding overfetching.
--
Ticket URL: <https://code.djangoproject.com/ticket/36606>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------

Reporter: Adam Johnson | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Adam Johnson:

Old description:

New description:

This case also seems untested with the `values_list()` tests in
`tests/lookup/tests.py`, so we'd want to add a test there.

--
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:1>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------

Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned

Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Adam Johnson):

* owner: (none) => Adam Johnson
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:2>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

I wonder if we should take the opportunity to deprecate
`values_list(flat=True)` instead by changing the check to be `len(fields)
!= 1` as it seems like an omission in the API and should be a rare
occurrence in code bases so I'm not convinced it's worth supporting it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:3>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Johnson):

Yes, I'm inclined to agree. It's not clear when reading what
`values_list(flat=True)` would do.

I think we can make the optimization *and* deprecate, though.
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:4>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Adam Johnson):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:5>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* stage: Unreviewed => Accepted

Comment:

Agree we should deprecate, happy to take a follow-up PR as Refs ... this
one.
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:6>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls <jacobtylerwalls@…>):

In [changeset:"2336d5d33a73032bd0d0745b21207ffc01f02b8e" 2336d5d]:
{{{#!CommitTicketReference repository=""
revision="2336d5d33a73032bd0d0745b21207ffc01f02b8e"
Refs #36606 -- Added tests for QuerySet.values_list(flat=True) without
fields.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:7>

[Django]

#36606: Optimize QuerySet.values_list(flat=True) with no fields
-------------------------------------+-------------------------------------
Reporter: Adam Johnson | Owner: Adam
Type: | Johnson

Cleanup/optimization | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls <jacobtylerwalls@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"2063c88c34566f46ad120c5b37c9926ffd3f10a6" 2063c88]:
{{{#!CommitTicketReference repository=""
revision="2063c88c34566f46ad120c5b37c9926ffd3f10a6"
Fixed #36606 -- Optimized QuerySet.values_list(flat=True) without fields.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36606#comment:8>