[Django] #33568: Random order doesn't work correctly with aggregation

6 views

Skip to first unread message

Django

unread,

Mar 8, 2022, 10:06:35 AM3/8/22

to django-...@googlegroups.com

#33568: Random order doesn't work correctly with aggregation
-------------------------------------+-------------------------------------
Reporter: Mohammad | Owner: nobody
Ali Mehdizadeh |
Type: Bug | Status: new
Component: Database | Version: 3.1
layer (models, ORM) |
Severity: Normal | Keywords: Random + Group By
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
This query:
Shop.objects\
.filter(Publish = True, Available = True)\
.annotate(product_count = Count('ShopProduct'))\
.filter(product_count__gt=1)\
.order_by('?').values('ID')[:12]

and ORM generate this SQL:
SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER JOIN
"nakhll_market_product" ON ("nakhll_market_shop"."ID" =
"nakhll_market_product"."FK_Shop_id") WHERE
("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
GROUP BY "nakhll_market_shop"."ID", RANDOM() HAVING
COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12

but it does not return any shop,
the correct SQL query is:
SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER JOIN
"nakhll_market_product" ON ("nakhll_market_shop"."ID" =
"nakhll_market_product"."FK_Shop_id") WHERE
("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
GROUP BY "nakhll_market_shop"."ID" HAVING
COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12

--
Ticket URL: <https://code.djangoproject.com/ticket/33568>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Mar 8, 2022, 10:24:42 AM3/8/22

to django-...@googlegroups.com

#33568: Random order doesn't work correctly with aggregation
-------------------------------------+-------------------------------------

Reporter: Mohammad Ali | Owner: nobody

Mehdizadeh |
Type: Bug | Status: new

Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:

Keywords: Random + Group By | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Tim Graham:

Old description:

> This query:
> Shop.objects\
> .filter(Publish = True, Available = True)\
> .annotate(product_count = Count('ShopProduct'))\
> .filter(product_count__gt=1)\
> .order_by('?').values('ID')[:12]
>
> and ORM generate this SQL:
> SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER
> JOIN "nakhll_market_product" ON ("nakhll_market_shop"."ID" =
> "nakhll_market_product"."FK_Shop_id") WHERE
> ("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
> GROUP BY "nakhll_market_shop"."ID", RANDOM() HAVING
> COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12
>
> but it does not return any shop,
> the correct SQL query is:
> SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER
> JOIN "nakhll_market_product" ON ("nakhll_market_shop"."ID" =
> "nakhll_market_product"."FK_Shop_id") WHERE
> ("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
> GROUP BY "nakhll_market_shop"."ID" HAVING
> COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12

New description:

This query:
{{{ #!python

Shop.objects\
.filter(Publish = True, Available = True)\
.annotate(product_count = Count('ShopProduct'))\
.filter(product_count__gt=1)\
.order_by('?').values('ID')[:12]
}}}
and ORM generate this SQL:

{{{ #!sql

SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER JOIN
"nakhll_market_product" ON ("nakhll_market_shop"."ID" =
"nakhll_market_product"."FK_Shop_id") WHERE
("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
GROUP BY "nakhll_market_shop"."ID", RANDOM() HAVING
COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12
}}}
but it does not return any shop,
the correct SQL query is:

{{{ #!sql

SELECT "nakhll_market_shop"."ID" FROM "nakhll_market_shop" LEFT OUTER JOIN
"nakhll_market_product" ON ("nakhll_market_shop"."ID" =
"nakhll_market_product"."FK_Shop_id") WHERE
("nakhll_market_shop"."Available" AND "nakhll_market_shop"."Publish")
GROUP BY "nakhll_market_shop"."ID" HAVING
COUNT("nakhll_market_product"."ID") > 1 ORDER BY RANDOM() ASC LIMIT 12
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33568#comment:1>

Django

unread,

Mar 8, 2022, 10:42:39 AM3/8/22

to django-...@googlegroups.com

#33568: Random order doesn't work correctly with aggregation
-------------------------------------+-------------------------------------
Reporter: Mohammad Ali | Owner: nobody
Mehdizadeh |

Type: Bug | Status: closed

Component: Database layer | Version: 3.1
(models, ORM) |

Severity: Normal | Resolution: duplicate