[Django] #31967: Django fails with FieldError: Expression contains mixed types: MoneyField, IntegerField. You must set output_field.
Django
IntegerField. You must set output_field.
-----------------------------------------+------------------------
Reporter: Matt Hegarty | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
This is related to #31919. Please see attached for full stack trace.
This is a regression because it does not occur in 3.1.
To reproduce:
{{{
git clone https://github.com/matthewhegarty/rest-framework-tutorial
cd rest-framework-tutorial
git checkout bug-expression-contains-mixed-types
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
./manage.py migrate
./manage.py runserver
}}}
browse to http://localhost:8000/snippets/
--
Ticket URL: <https://code.djangoproject.com/ticket/31967>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
IntegerField. You must set output_field.
Reporter: Matt Hegarty | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Matt Hegarty):
* Attachment "stack_trace.txt" added.
stack trace
Django
IntegerField. You must set output_field.
Reporter: Matt Hegarty | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Matt Hegarty):
* cc: Matt Hegarty (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:1>
Django
IntegerField. You must set output_field.
Reporter: Matt Hegarty | Owner: nobody
Component: Database layer | Version: master
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* cc: Simon Charette (added)
* type: Uncategorized => Bug
* component: Uncategorized => Database layer (models, ORM)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
Thanks for this report. It's a regression in
1e38f1191de21b6e96736f58df57dfb851a28c1f but it's not related with fields
subclasses. I was able to reproduce it with a builtin `DecimalField()`,
e.g.
{{{
>>> list(MyModel.objects.annotate(x=Coalesce(F('field_decimal'), 0))
...
File "django/django/db/models/expressions.py", line 306, in
_resolve_output_field
source.__class__.__name__,
django.core.exceptions.FieldError: Expression contains mixed types:
DecimalField, IntegerField. You must set output_field.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: nobody
Component: Database layer | Version: master
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:3>
Django
Reporter: Matt Hegarty | Owner: nobody
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* component: Database layer (models, ORM) => Documentation
* severity: Release blocker => Normal
Comment:
We didn't resolve output fields for `Value()` expressions before
1e38f1191de21b6e96736f58df57dfb851a28c1f, so they were ignored when
resolving an output field for functions. Now, we take them into account,
that's why an error is raised. IMO it's an expected behavior. It should be
enough to mention this in release notes.
You can fix this by using `Decimal('0')`, e.g.
`Coalesce(F('field_decimal'), Decimal('0')))`
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:4>
Django
-------------------------------+------------------------------------
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Matt Hegarty):
When I try to set the second arg to Decimal('0'), I get:
{{{
django.db.utils.ProgrammingError: can't adapt type 'DecimalField'
}}}
Also, I think this workaround will be confusing, because the error message
tells you to set the output_field. I did try this when I first
encountered the error and it didn't work. So it's very likely that others
will do the same rather than checking the release notes.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:5>
Django
-------------------------------+------------------------------------
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by felixxm):
Replying to [comment:5 Matt Hegarty]:
> When I try to set the second arg to Decimal('0'), I get:
>
> {{{
> django.db.utils.ProgrammingError: can't adapt type 'DecimalField'
> }}}
>
True, sorry, setting `output_field` is the best way.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:6>
Django
-------------------------------+------------------------------------
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Matt Hegarty):
I confirm that this call does not crash:
{{{
Coalesce(
SubquerySum(
"price__price"
),
0,
output_field=DecimalField()
),
}}}
Thank you for your help.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:7>
Django
-------------------------------+------------------------------------
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Simon Charette):
So we've got two approach we could take here.
Determine that it was never meant to work and document it or use a
clemency based approach where we deprecate this behaviour by ignoring the
`output_field` of `Value` source expressions in
`Expression._resolve_output_field` when it fails with `FieldError`.
I'm surprised `Coalesce(F('field_decimal'), Decimal('0')))` doesn't
actually work.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:8>
Django
-------------------------------+------------------------------------
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by felixxm):
I think it's enough to document this change in release notes, because it's
easy to fix on the users side, it's explicit, and it was never meant to
work.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:9>
Django
Reporter: Matt Hegarty | Owner: Hasan Ramezani
Type: Bug | Status: assigned
Component: Documentation | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Hasan Ramezani):
* owner: nobody => Hasan Ramezani
* status: new => assigned
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:10>
Django
Reporter: Matt Hegarty | Owner: Hasan
| Ramezani
Type: Bug | Status: assigned
Component: Documentation | Version: master
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* stage: Accepted => Ready for checkin
Comment:
[https://github.com/django/django/pull/13400 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Hasan
| Ramezani
Component: Documentation | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"9369f0cebba1f65909a14dec6aa3515ec1eb2557" 9369f0ce]:
{{{
#!CommitTicketReference repository=""
revision="9369f0cebba1f65909a14dec6aa3515ec1eb2557"
Fixed #31967 -- Doc'd consequences of resolving an output_field for
Value().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Hasan
| Ramezani
Type: Bug | Status: closed
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by retpolanne):
Hello! I see that this is fixed, but I still don't understand how to
workaround this issue.
I'm working on updating a Django app using this library:
https://github.com/annemacedo-tw/django-spillway
I get this error when using it:
{{{
django.core.exceptions.FieldError: Expression contains mixed types:
MultiPolygonField, FloatField. You must set output_field.
}}}
However I don't get the same error on Django 3.1
As far as I debugged, this line here yields these two fields
https://github.com/annemacedo-tw/django-
spillway/blob/fa008ef0db5f3434c1ef438f66231f35a90b1e6d/spillway/query.py#L153
{{{
SimplifyPreserveTopology(F(geometry), Value(0.0439453125))
}}}
The first one being a MultiPolygonField and the other one being a
FloatField. I tried setting the output_field and recreating the
source_expressions but it didn't really work.
{{{
sql = SimplifyPreserveTopology(sql, tilew)
sql_field = sql.source_expressions[0]
sql_value = sql.source_expressions[1]
sql_value.output_field = models.MultiPolygonField()
sql.set_source_expressions([sql_field, sql_value])
}}}
(I understand that I'm setting an output field for MultiPolygonField here
instead of FloatField as should be expected, I set it up to avoid the
mixed types error)
{{{
File "/usr/local/lib/python3.9/site-
packages/django/contrib/gis/db/models/fields.py", line 194, in
get_prep_value
raise ValueError('Cannot use object with type %s for a spatial lookup
parameter.' % type(obj).__name__)
ValueError: Cannot use object with type float for a spatial lookup
parameter.
}}}
TL;DR how can I work around this issue when using GeoDjango?
Another thing I noticed is that ever since GeoManager was deprecated way
back in 1.10, django.contrib.gis.db.models.FloatField resolves to
django.db.models.fields.FloatField. I wonder if the gis FloatField would
play nicely with MultiPolygonField.
{{{
(Pdb) from django.contrib.gis.db import models
(Pdb) models.FloatField
<class 'django.db.models.fields.FloatField'>
}}}
I wish I could open a separate ticket for this, but I'm working on legacy
code that I don't understand so well, so I don't know how to do a minimal
version out of it.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Hasan
| Ramezani
Type: Bug | Status: closed
Component: Documentation | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Simon Charette):
I suggest posting your question in the
[https://forum.djangoproject.com/c/internals/geodjango/13 GeoDjango
category of the forum] instead.
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Hasan
| Ramezani
Type: Bug | Status: closed
Component: Documentation | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Anne "Anya" Macedo):
Replying to [comment:14 Simon Charette]:
> I suggest posting your question in the
[https://forum.djangoproject.com/c/internals/geodjango/13 GeoDjango
category of the forum] instead.
>
> I suspect you only have to pass
`output_field=models.MultiPolygonField()` when initializing your
`SimplifyPreserveTopology` as hinted by the error message: `You must set
output_field.`
>
> {{{#!python
> SimplifyPreserveTopology(
> F(geometry), 0.0439453125, output_field=models.MultiPolygonField()
> )
> }}}
I believe it worked! Been trying to fix this bug for a whole week :)
--
Ticket URL: <https://code.djangoproject.com/ticket/31967#comment:15>