[Django] #36426: prefetch_related_objects() is doc'd to accept iterables but only accepts sequences

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
The docs say `prefetch_related_objects()`
[https://docs.djangoproject.com/en/5.2/ref/models/querysets/#django.db.models.prefetch_related_objects
accepts iterables], but it only accepts sequences.

{{{#!py
In [1]: from django.db.models import prefetch_related_objects

In [2]: objs = set(Graph.objects.all())

In [3]: prefetch_related_objects(objs, "node_set")
---------------------------------------------------------------------------
TypeError Traceback (most recent call
last)
Cell In[3], line 1
----> 1 prefetch_related_objects(objs, "node_set")

File ~/py313/lib/python3.13/site-packages/django/db/models/query.py:2361,
in prefetch_related_objects(model_instances, *related_lookups)
2355 break
2357 # Descend down tree
2358
2359 # We assume that objects retrieved are homogeneous (which is the
premise
2360 # of prefetch_related), so what applies to first object applies to
all.
-> 2361 first_obj = obj_list[0]
2362 to_attr = lookup.get_current_to_attr(level)[0]
2363 prefetcher, descriptor, attr_found, is_fetched = get_prefetcher(
2364 first_obj, through_attr, to_attr
2365 )

TypeError: 'set' object is not subscriptable
}}}

We could adjust the docs, but at a glance we could probably instead adjust
the implementation to just `next()` instead of `[0]` and accept iterables
as documented.

django-stubs [https://github.com/typeddjango/django-
stubs/blob/5bb841674610e88a162475b7ecf2e950a92c23a8/django-
stubs/db/models/query.pyi#L247 types] the argument as `Model[Iterable]`.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------

Reporter: Jacob Walls | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Jacob Walls:

Old description:

New description:

stubs/db/models/query.pyi#L247 types] the argument as `Iterable[_Model]`.

--
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:1>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------

Reporter: Jacob Walls | Owner: Luna
Type: Bug | Status: assigned

Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Luna):

* owner: (none) => Luna
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:2>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna

Type: | Status: assigned
Cleanup/optimization |

Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* stage: Unreviewed => Accepted
* type: Bug => Cleanup/optimization

Comment:

Thank you Jacob for creating this ticket!

My suggestion would be to adjust the docs, since there are `Iterable`
types that I think make less sense to support in this context, for
example, `str` or `dict`. Also, by definition, an `Iterable` only needs to
implement `__iter__`, which means it wouldn't necessarily support
`__next__` (that would make it an `Iterator`). The difference is subtle,
though, and even a `list`, while `Iterable`, is not an `Iterator`.

I'd suggest we update the documentation to clarify that the argument
should be a sequence of model instances, like a list or tuple, rather than
any arbitrary iterable. For example:
{{{
Prefetches the given lookups on a sequence of model instances, such as a
list or tuple.
}}}
This keeps the expectations clear and avoids surprising behavior with edge
cases like passing a string or dictionary.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:3>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

Sorry, I didn't realize I left out details: I figured
`next(iter(model_instances)))` would accept iterables simply.

> This keeps the expectations clear and avoids surprising behavior with
edge cases like passing a string or dictionary.

Certainly a string makes no sense, but a string is both an iterable and a
sequence, so it's the same no matter what we choose here. I'm not
proposing we add a strict type check for model instances: the current
duck-typing approach seems fine.

I used `set` in the bug report, but where I encountered this in practice
was with a `dict_values`. I could have just as easily ended up trying
`dict` or `dict_keys`. I guess expected them all to just work.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:4>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by bistasulove):

Hi Jacob and Natalia, is the documentation fix being worked on? I can
raise the PR otherwise. Let me know, thanks
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:5>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

This ticket is assigned to someone. Let's allow them some more time. We
don't yet have clear acceptance of the idea to accept iterables, but since
the suggested change is small and is a pattern already used in the
project, seeing a tested PR might help with decision-making.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:6>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |

Component: Database layer | Version: dev

(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* version: 5.2 => dev

Comment:

Replying to [comment:4 Jacob Walls]:

> Sorry, I didn't realize I left out details: I figured
`next(iter(model_instances)))` would accept iterables simply.

Makes sense :-)

> > This keeps the expectations clear and avoids surprising behavior with
edge cases like passing a string or dictionary.
>
> Certainly a string makes no sense, but a string is both an iterable and
a sequence, so it's the same no matter what we choose here.

Good point.

> I'm not proposing we add a strict type check for model instances: the
current duck-typing approach seems fine.

I agree.

> I used `set` in the bug report, but where I encountered this in practice
was with a `dict_values`. I could have just as easily ended up trying
`dict` or `dict_keys`. I guess expected them all to just work.

Your points make sense and I think changing the code (with tests) to
replace `first_obj = obj_list[0]` with `next(iter(obj_list))` makes the
code more robust, and as you say, it aligns with the docs.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:7>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Luna):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:8>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Luna):

Hello,

I have just submitted a PR related to this ticket for review. Following
the suggested approach, the patch modifies the implementation to accept
any iterable by using `next(iter(...))` instead of indexing. This change
aligns with the documentation and improves code robustness.

Please let me know if there are any further improvements or feedback
needed.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:9>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:10>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by JaeHyuckSa):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:11>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_better_patch: 0 => 1

Comment:

Not all PR comments have been addressed
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:12>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by JaeHyuckSa):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:13>

[Django]

#36426: prefetch_related_objects() is doc'd to accept iterables but only accepts
sequences
-------------------------------------+-------------------------------------
Reporter: Jacob Walls | Owner: Luna
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by JaeHyuckSa):

* needs_better_patch: 0 => 1

Comment:

I didn’t find any further changes.
--
Ticket URL: <https://code.djangoproject.com/ticket/36426#comment:14>