select_related() / prefetch_related() and sharding

[David]

Dear List,

I came about conflicting information (to my mind) concerning the use of
select_related() and prefetch_related().
My question is whether you can combine select_related() and
prefetch_related() with sharding techniques. If so, how are you doing it?

Background:
I was watching various videos on Django under load and Django scaling,
generally there comes a point at which they point at database sharding.
Ash Christopher (https://www.youtube.com/watch?v=frlsGUHYu04) points out
that as you use multiple databases with your Application,
select_related() and prefetch_related() can no longer be used. In
Christophe Pettus' videos I have not found such a comment, in fact
select_related() and prefetch_related() seem to be habitually
recommended to speed database connections up.

Thanks for your insights!

David

[Ash Christopher]

Hi David,

select_related does a JOIN under the hood, and you can't do cross-database joins in the databases supported by Django. For prefetch_related, there is no way out of the box to perform a cross-database join in Python (which is what prefetch_related is doing) but perhaps there is some way to programatically make that happen. I haven't put any effort into exploring that idea.

Thanks,

[Simon Charette]

Small correction, prefetch_related doesn't issue any JOIN unless a is query with a select_related is specified with a Prefetch object.

Simon