[Django] #30542: Float-valued aggregations and annotations with filters fail with AttributeError
Django
AttributeError
-------------------------------------+-------------------------------------
Reporter: Chuan- | Owner: nobody
Zheng Lee |
Type: Bug | Status: new
Component: Database | Version: 2.2
layer (models, ORM) | Keywords: aggregation,
Severity: Normal | annotation, filter
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When any float-valued aggregation or annotation (`Avg`, `StdDev`,
`Variance`) is used with the `filter=` keyword argument, the following
exception is raised:
{{{
AttributeError: 'WhereNode' object has no attribute 'output_field'
}}}
For example, these queries all fail with this error:
{{{#!python
Speaker.objects.annotate(average=Avg('speakerscore__score',
filter=Q(speakerscore__ghost=False)))
Speaker.objects.annotate(average=StdDev('speakerscore__score',
filter=Q(speakerscore__ghost=False)))
Team.objects.annotate(average=Avg('debateteam__teamscore__score',
filter=Q(debateteam__teamscore__forfeit=False)))
}}}
The error seems to be raised irrespective of what's in the database
(''e.g.'', it's raised even for an empty database). However, it doesn't
affect aggregations like `Sum`, `Max` or `Min` that don't use
`NumericOutputFieldMixin`. Also, queries that don't use the `filter=`
keyword in the aggregation work fine.
The exception in question is raised from line 46 of
django/db/models/functions/mixins.py, which looks for an `output_field`
attribute of every element of `self.get_source_expressions()`, where
`self` is the object containing `NumericOutputFieldMixin`, in this case
`Avg` or some other subclass of `Aggregate`. But
`Aggregate.get_source_expressions()` includes `self.filter` in its list,
and (post-compilation) `self.filter` is a `WhereNode`, which doesn't have
an `output_field` attribute.
This issue is new in version 2.2. Everything works fine in version 2.1
(where I believe `NumericOutputFieldMixin` didn't exist, or at least
wasn't on the MRO for `Avg`, `StdDev` or `Variance`).
=== Minimal reproducible example
In a blank (or any) project, insert in models.py:
{{{#!python
from django.db.models import Model, FloatField, BooleanField
class Book(Model):
price = FloatField()
fiction = BooleanField()
}}}
then run migrations and in `python manage.py shell`:
{{{
$ python manage.py shell
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django22_output_field_bug.models import Book
>>> from django.db.models import Avg, Q
>>> Book.objects.annotate(average=Avg('price', filter=Q(fiction=True)))
Traceback (most recent call last):
[...]
File "/[...]/django/db/models/functions/mixins.py", line 46, in
<genexpr>
if any(isinstance(s.output_field, DecimalField) for s in
source_expressions):
AttributeError: 'WhereNode' object has no attribute 'output_field'
>>> Book.objects.aggregate(average=Avg('price', filter=Q(fiction=True)))
Traceback (most recent call last):
[...]
File "/[...]/django/db/models/functions/mixins.py", line 46, in
<genexpr>
if any(isinstance(s.output_field, DecimalField) for s in
source_expressions):
AttributeError: 'WhereNode' object has no attribute 'output_field'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30542>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
AttributeError
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: aggregation, | Triage Stage:
annotation, filter | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chuan-Zheng Lee):
I couldn't tell you for sure because I haven't actually tried reproducing
it on this commit or its parent, but if I had to take a guess as to which
commit introduced the issue, I'd try
a0b19a0f5b1731cf575546175034da53f5af5367, which introduced
`OutputFieldMixin`, a precursor to `NumericOutputFieldMixin` with similar
logic. I'll say something if I manage to give this a go.
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:1>
Django
AttributeError
-------------------------------------+-------------------------------------
| Beaulé
Type: Bug | Status: assigned
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
annotation, filter |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => assigned
* owner: nobody => Étienne Beaulé
* has_patch: 0 => 1
* version: 2.2 => 2.0
* stage: Unreviewed => Accepted
Comment:
[https://github.com/django/django/pull/11443 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:2>
Django
AttributeError
-------------------------------------+-------------------------------------
| Beaulé
Type: Bug | Status: assigned
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: aggregation, | Triage Stage: Accepted
annotation, filter |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chuan-Zheng Lee):
Contrary to my earlier guess, the first commit with the issue is
c690afb873cac8035a3cb3be7c597a5ff0e4b261, which modifies `Avg` to use the
new logic. Its parent, 3d5e0f8394688d40036e27cfcfac295e6fe622609, works
fine, because while it modified the mixin to its current form, it didn't
touch `Avg`.
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:3>
Django
AttributeError
-------------------------------------+-------------------------------------
| Beaulé
Type: Bug | Status: assigned
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: aggregation, | Triage Stage: Accepted
annotation, filter |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 2.0 => 2.2
* severity: Normal => Release blocker
Comment:
Correct, this is a regression in
[https://github.com/django/django/pull/10764/commits/c690afb873cac8035a3cb3be7c597a5ff0e4b261
c690afb] which affects `Avg()`. `StdDev()` and `Variance()` are affected
by
[https://github.com/django/django/pull/10764/commits/e85afa5943695457c85e9bc1c5dc0d985004e303
e85afa5] and
[https://github.com/django/django/pull/10764/commits/6d4efa8e6a4cc7be4ba957dec71f6f63cd58700d
6d4efa8] respectively.
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:4>
Django
AttributeError
-------------------------------------+-------------------------------------
| Beaulé
Component: Database layer | Version: 2.2
(models, ORM) |
Keywords: aggregation, | Triage Stage: Accepted
annotation, filter |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"4b6dfe16226a81fea464ac5f77942f4d6ba266e8" 4b6dfe1]:
{{{
#!CommitTicketReference repository=""
revision="4b6dfe16226a81fea464ac5f77942f4d6ba266e8"
Fixed #30542 -- Fixed crash of numerical aggregations with filter.
Filters in annotations crashed when used with numerical-type
aggregations (i.e. Avg, StdDev, and Variance). This was caused as the
source expressions no not necessarily have an output_field (such as the
filter field), which lead to an AttributeError: 'WhereNode' object has
no attribute output_field.
Thanks to Chuan-Zheng Lee for the report.
Regression in c690afb873cac8035a3cb3be7c597a5ff0e4b261 and two following
commits.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:5>
Django
AttributeError
-------------------------------------+-------------------------------------
| Beaulé
Type: Bug | Status: closed
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: aggregation, | Triage Stage: Accepted
annotation, filter |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"4e6f0024f19f74be2fe4d9ec637b87e82054ebd6" 4e6f002]:
{{{
#!CommitTicketReference repository=""
revision="4e6f0024f19f74be2fe4d9ec637b87e82054ebd6"
[2.2.x] Fixed #30542 -- Fixed crash of numerical aggregations with filter.
Filters in annotations crashed when used with numerical-type
aggregations (i.e. Avg, StdDev, and Variance). This was caused as the
source expressions no not necessarily have an output_field (such as the
filter field), which lead to an AttributeError: 'WhereNode' object has
no attribute output_field.
Thanks to Chuan-Zheng Lee for the report.
Regression in c690afb873cac8035a3cb3be7c597a5ff0e4b261 and two following
commits.
Backport of 4b6dfe16226a81fea464ac5f77942f4d6ba266e8 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30542#comment:6>