[Django] #33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related

2 views
Skip to first unread message

Django

unread,
Feb 4, 2022, 1:10:18 PM2/4/22
to django-...@googlegroups.com
#33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related
-----------------------------------------+------------------------
Reporter: abetkin | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
Hi django fellows!

I've stumbled upon this code in the main branch: (see screenshot)

The usecase is iterating through a queryset with prefetch_related lookups
specified.

Do I understand correctly that multiple database queries happen upon
fetching every chunk, and also that multiple database connections are
required?

If so, than I think it would be better just to prohibit the use of
prefetch_related with QuerySet.iterate

--
Ticket URL: <https://code.djangoproject.com/ticket/33493>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Feb 4, 2022, 1:10:36 PM2/4/22
to django-...@googlegroups.com
#33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related
-------------------------------+--------------------------------------

Reporter: abetkin | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by abetkin):

* Attachment "methods_that.png" added.

Django

unread,
Feb 4, 2022, 1:15:27 PM2/4/22
to django-...@googlegroups.com
#33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related
-------------------------------------+-------------------------------------
Reporter: abetkin | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by abetkin):

* type: Uncategorized => Bug
* component: Uncategorized => Database layer (models, ORM)


--
Ticket URL: <https://code.djangoproject.com/ticket/33493#comment:1>

Django

unread,
Feb 4, 2022, 1:15:58 PM2/4/22
to django-...@googlegroups.com
#33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related
-------------------------------------+-------------------------------------
Reporter: abetkin | Owner: nobody

Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by abetkin):

* stage: Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/33493#comment:2>

Django

unread,
Feb 4, 2022, 2:10:16 PM2/4/22
to django-...@googlegroups.com
#33493: It's counter-intuitive how QuerySet.iterate() works with prefetch_related
-------------------------------------+-------------------------------------
Reporter: abetkin | Owner: nobody
Type: | Status: closed
Cleanup/optimization |

Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* status: new => closed
* type: Bug => Cleanup/optimization
* resolution: => wontfix


Comment:

Support for prefetching related objects with `QuerySet.iterator()` was
added in edbf930287cb72e9afab1f7208c24b1146b0c4ec, and yes an extra query
is necessary for each chunk, it's documented. You can check related
discussions in the ticket #29984, on the [https://groups.google.com/g
/django-developers/c/ADgUd6jRvdw/m/_ZnETNlcAAAJ mailing list], and in
[https://github.com/django/django/pull/15334 PR]. I don't see any reason
to remove this feature.

--
Ticket URL: <https://code.djangoproject.com/ticket/33493#comment:3>

Reply all
Reply to author
Forward
0 new messages