Worth raising a warning when `order_by('?')` is used with specific backends?

[Santiago Basulto]

Sadly I have to admit that I'm not involved with day to day development of our app anymore (I miss it tremendously). Yesterday I felt nostalgic and reviewed a few already-merged PRs, just "for fun" we could say. Great was my surprise when I noticed that one of those PRs was merged with an `order_by('?')`. We're (and we've always been) using Postgres, and I knew already that `ORDER BY RANDOM()` is very slow. I just submitted a comment and it's already been fixed, so it wasn't a big deal. But this got me thinking:

TLDR;

Some less-experienced developers might not know the issues with `order_by('?')` (at least with Postgres, the only engine I'm familiar enough) and it's worth raising a warning. I don't know what's the state of other engines, it's maybe even worth raising the issue for ANY engine.

It could help newcomers catch these issues early. The docs clearly state it, but not everybody reads the docs with such level of detail:

Note: order_by('?') queries may be expensive and slow, depending on the database backend you’re using

PS: Sorry for the personal touch of this message, but I think it's worth the explanation.

[Adam Johnson]

Hi Santiago

There are any kinds of queries that can be slow in Django. ORDER BY RANDOM() will be just as slow on large tables as ORDER BY some_unindexed_column, or many other SQL constructs.

Django can't predict well how expensive a query will be - that's the job of the database's optimizer. It varies on many factors that only the database server knows - table size, index size, column histograms, etc.

I therefore don't think such a warning would be very accurate or useful.

(Also, just as documentation can be missed, so too can warnings.)

Thanks,

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/fe948733-7767-4688-b89a-43fa57153905%40googlegroups.com.

--

Adam

[Tim Graham]

I think issuing a warning would be rude to developers who want to use that feature and are aware of the limitation... they would have to do extra work to silence the warning.

[Santiago Basulto]

👍 ok, great. They're both good points. As to Adam's point about missing warnings: it is true, they can be missed, but there are ways to enforce them (python -W error, or PYTHONWARNINGS=error).

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/WdkpBmN6XRc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d11f0f8c-a73f-4e95-9b69-25d3195effb8%40googlegroups.com.

--

Santiago Basulto.-

Up!