[Django] #32200: Aggregating when grouping on an ExpressionWrapper omits the expression from the group by
Django
the group by
------------------------------------------+------------------------
Reporter: Gordon Wrigley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------+------------------------
I ran into this with Postgres on Django 3.1.3, I'm not sure what other
versions it exists on.
{{{
#!python
print(
Fred.objects.annotate(
bob_id__is_null=ExpressionWrapper(
Q(bob_id=None),
output_field=BooleanField()
)
).values(
"bob_id__is_null"
).annotate(
id__count=Count("id", distinct=True)
).values(
"bob_id__is_null",
"id__count"
).query
)
}}}
{{{
#!sql
SELECT
"main_fred"."bob_id" IS NULL AS "bob_id__is_null",
COUNT(DISTINCT "main_fred"."id") AS "id__count"
FROM "main_fred"
GROUP BY "main_fred"."bob_id"
}}}
On the last line there the group by has dropped the "IS NULL"
--
Ticket URL: <https://code.djangoproject.com/ticket/32200>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Gordon Wrigley):
For anyone else who ends up here you can Subquery your way out of this,
although I've no idea what the performance implications of that are.
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:1>
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Gordon Wrigley):
I did some further digging and this only occurs on 3.1, it works fine on
2.0, 2.1, 2.2 and 3.0
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:2>
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Gordon Wrigley):
I think this is related to https://code.djangoproject.com/ticket/32007
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:3>
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.1
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Gordon Wrigley):
To test this, given the implied model above, you can create 3 Fred
objects, 2 with one value for bob_id and the third with a different value.
When you do the select on that you should see `[{"bob_id__is_null": False,
"id_count": 3}]`
But instead you will get `[{"bob_id__is_null": False, "id_count": 1},
{"bob_id__is_null": False, "id_count": 2}]`
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:4>
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* type: Uncategorized => Bug
* component: Uncategorized => Database layer (models, ORM)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
Regression in df32fd42b84cc6dbba173201f244491b0d154a63 (backported in
fdd2b01e8e12857aad2219a46a41bd9051ec8f8d).
Reproduced at 4cce1d13cfe9d8e56921c5fa8c61e3034dc8e20c.
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:5>
Django
the group by
Reporter: Gordon Wrigley | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* cc: Simon Charette, Thodoris Sotiropoulos (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:6>
Django
the group by
Reporter: Gordon Wrigley | Owner: Hasan
| Ramezani
Type: Bug | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Hasan Ramezani):
* owner: nobody => Hasan Ramezani
* status: new => assigned
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:7>
Django
the group by
Reporter: Gordon Wrigley | Owner: Hasan
| Ramezani
Type: Bug | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 0 => 1
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:8>
Django
the group by
Reporter: Gordon Wrigley | Owner: Hasan
| Ramezani
Type: Bug | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:9>
Django
the group by
Reporter: Gordon Wrigley | Owner: Hasan
| Ramezani
Component: Database layer | Version: 3.1
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"fe9c7ded2996364f853c524b4421274717d89d5f" fe9c7ded]:
{{{
#!CommitTicketReference repository=""
revision="fe9c7ded2996364f853c524b4421274717d89d5f"
Fixed #32200 -- Fixed grouping by ExpressionWrapper() with Q objects.
Thanks Gordon Wrigley for the report.
Regression in df32fd42b84cc6dbba173201f244491b0d154a63.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:10>
Django
the group by
Reporter: Gordon Wrigley | Owner: Hasan
| Ramezani
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"166c0d2474a3b5e09e1dd96ec93bdbf08f0c65c2" 166c0d24]:
{{{
#!CommitTicketReference repository=""
revision="166c0d2474a3b5e09e1dd96ec93bdbf08f0c65c2"
[3.1.x] Fixed #32200 -- Fixed grouping by ExpressionWrapper() with Q
objects.
Thanks Gordon Wrigley for the report.
Regression in df32fd42b84cc6dbba173201f244491b0d154a63.
Backport of fe9c7ded2996364f853c524b4421274717d89d5f from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/32200#comment:11>