Prefetching returning empty values for some object

29 views
Skip to first unread message

Mohit Solanki

unread,
Dec 30, 2019, 2:20:07 AM12/30/19
to django...@googlegroups.com
Assume these two models.

class Folder(model):
    name = models.CharField(max_length=8)
    files = models.ManyToManyField('File')

class File:
   path = models.CharField(max_length=255)

First I was fetching the folders and while looping over them fetching the files with a distinct path

folders = Folder.objects.all()

for folder in folders:
    files_with_distinct_path = folder.files.distinct('path').order_by('path')

but this suffers from the N+1 problem, So I tried prefetching the files while fetching all the folders.

from django.db.models import Prefetch

folders = Folder.objects.prefetch_related(
    Prefetch(
        'files', queryset=File.objects.distinct('path').order_by('path'), to_attr='distinct_files'
    )
)

but it turns out now there objects with empty `distinct_files` which is not the correct result, these objects have files but after prefetching, the result is empty only for few objects.

Simon Charette

unread,
Dec 30, 2019, 10:40:02 AM12/30/19
to Django users
I suspect your usage of `distinct('path')` is causing issues during prefetching here
as it will prevent different File with the same .path from being prefetched for two different
folder objects.

You likely want to use `distinct('path', 'folder')` here instead where 'folder' might have a different
name based on whether or not you provided a related_name or related_query_name to Folder.files.

Cheers,
Simon
Reply all
Reply to author
Forward
0 new messages