Hello Tobias,
From my understanding the introduction of chunk_size doest't help here.
The fundamental reason why iterator() cannot be used with prefetch_related() is that the latter requires a set of model instance to be materialized to work appropriately which chunk_size doesn't control at all.
In other words chunk_size only controls how many rows should be fetched from the database cursor and kept into memory at a time. Even when this parameter is used, iterator() will only materialize a single model instance per yield.
Now, something you could do if you really want to be using prefetch_related with iterator() is to materialize chunks and use the now public prefetch_related_objects[0] helper to prefetch relationships.
A somewhat complete implementation would look like
def prefetch_related_iterator(queryset, *related_lookups, chunk_size=100):
iterator = queryset.iterator(chunk_size=chunk_size)
while True:
chunk = list(itertools.islice(iterator, chunk_size))
if not chunk:
return
prefetch_related_objects(chunk, *related_lookups)
yield from chunk
Given that iterator() always ignored prefetch related lookups instead of erroring out when they were specified make me feel like turning such a feature on by default could be problematic as it could balloon the memory usage which is the main reason why iterator is useful anyway.
Cheers,
Simon