As a follow-up to my last message, there seems to be another issue
when the foreign key is optional. If you set 'null=True, blank=True'
on the host field of the accounts model above, Django joins everything
except the foreign key and then hits the database every time it needs
data relating to the foreign key. The result is another O(m*n)
database hits where m is the number of foreign keys displayed by the
admin model and n is the number of rows being rendered in the
change_list. These hits would be unnecessary if Django were to use a
LEFT OUTER JOIN when rendering potentially null foreign keys. To
continue with the example above, the queries are now:
EXECUTIONS | TIME | QUERY
40 | 396 ms | SELECT "app_host"."id", "app_host"."name" FROM
40 | 331 ms | SELECT "app_host"."id", "app_host"."name" FROM
"app_host" WHERE "app_host"."id" = %s
----> would not be necessary if the main query
(below) used a LEFT OUTER JOIN
1 | 15 ms | SELECT COUNT(*) FROM "app_account"
1 | 10 ms | SELECT "auth_message"."id", "auth_message"."user_id",
"auth_message"."message" FROM "auth_message" WHERE
"auth_message"."user_id" = %s
1 | 0 ms | SELECT "app_account"."id", "app_account"."host_id",
"app_account"."name" FROM "app_account" ORDER BY "app_account"."id"
----- >should include a LEFT OUTER JOIN on app_host
Again I realize that this might be my mistake, so any input would be
Also, as a follow up to my last email, I think that that is indeed a
bug in Django. Not only does the documentation list a similar issue
for using list_display on ManyToMany fields, but I have also managed
to create two fixes (hacks?) that address the problem:
1. Adding a name-based cache to the Select widget rendering function
(django\forms\widgets.py:~431). This fix works, but it is clearly less
than ideal for its lack of flexibility and the fact that it does not
address the underlying issue.
2. Manually rendering the data like a widget in admin.py using a
custom display function that caches the data while rendering the first
row. This also works but again fails to address the underlying issue.
Given that these two fixes work, I am fairly confident that a better
solution is out there. Maybe someone who knows a little more about how
Django decides how to build and cache database queries could help me
figure this out?