[Django] #36574: Regression: DecimalField values are no longer quantized before written to the DB

[Django]

#36574: Regression: DecimalField values are no longer quantized before written to
the DB
-------------------------------------+-------------------------------------
Reporter: Aaron Mader | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Steps to reproduce:
1. Create a new project using the mysql backend
2. Create this model:
{{{#!python
class Apple(models.Model):
weight = models.DecimalField(max_digits=4, decimal_places=1)
}}}
3. Create an instance of this record with input outside of the
quantization limits
{{{#!python
from decimal import Decimal
Apple.objects.create(
weight = Decimal("1.01"),
)
}}}
4. Attempt to fetch the record by the (expected) adjusted value
{{{#!python
from decimal import Decimal
Apple.objects.filter(
weight = Decimal("1.0"),
)
}}}

Specifically, this change in behaviour was introduced in django 5.2.0,
with prior versions of django quantizing the input value (`1.01`) to an
acceptable value (`1.0`) before writing the value to the database.
I believe that change was introduced in pull request:
https://github.com/django/django/pull/18895
--
Ticket URL: <https://code.djangoproject.com/ticket/36574>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36574: Regression: DecimalField values are no longer quantized before written to
the DB
-------------------------------------+-------------------------------------

Reporter: Aaron Mader | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Aaron Mader:

Old description:

> Steps to reproduce:
> 1. Create a new project using the mysql backend
> 2. Create this model:
> {{{#!python
> class Apple(models.Model):
> weight = models.DecimalField(max_digits=4, decimal_places=1)
> }}}
> 3. Create an instance of this record with input outside of the
> quantization limits
> {{{#!python
> from decimal import Decimal
> Apple.objects.create(
> weight = Decimal("1.01"),
> )
> }}}
> 4. Attempt to fetch the record by the (expected) adjusted value
> {{{#!python
> from decimal import Decimal
> Apple.objects.filter(
> weight = Decimal("1.0"),
> )
> }}}
>
> Specifically, this change in behaviour was introduced in django 5.2.0,
> with prior versions of django quantizing the input value (`1.01`) to an
> acceptable value (`1.0`) before writing the value to the database.
> I believe that change was introduced in pull request:
> https://github.com/django/django/pull/18895

New description:

Steps to reproduce:
1. Create a new project using the mysql backend
2. Create this model:
{{{#!python
class Apple(models.Model):
weight = models.DecimalField(max_digits=4, decimal_places=1)
}}}
3. Create an instance of this record with input outside of the
quantization limits
{{{#!python
from decimal import Decimal
Apple.objects.create(
weight = Decimal("1.01"),
)
}}}
4. Attempt to fetch the record by the (expected) adjusted value
{{{#!python
from decimal import Decimal
Apple.objects.filter(
weight = Decimal("1.0"),
)
}}}

Result:
With django 5.2.0, the filter query fails with 0 results.
With django 5.1.11, the filter query success, returning the created
record.

Specifically, this change in behaviour was introduced in django 5.2.0,
with prior versions of django quantizing the input value (`1.01`) to an
acceptable value (`1.0`) before writing the value to the database.
I believe that change was introduced in pull request:
https://github.com/django/django/pull/18895

--

--
Ticket URL: <https://code.djangoproject.com/ticket/36574#comment:1>

[Django]

#36574: Regression: DecimalField values are no longer quantized before written to
the DB
-------------------------------------+-------------------------------------
Reporter: Aaron Mader | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

I'm not sure that the old behavior of silently discarding decimal places
and matching another value is justified. Could you explain your use case?
--
Ticket URL: <https://code.djangoproject.com/ticket/36574#comment:2>

[Django]

#36574: Regression: DecimalField values are no longer quantized before written to
the DB
-------------------------------------+-------------------------------------
Reporter: Aaron Mader | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

I also share Tim's position; if input validation is not performed then
it's down to how the database backend implements storeage (e.g. SQLite
will allow to store pretty much anything in any field)

Do you happen [https://dbfiddle.uk/DiVEJQCN to be running MySQL in non-
strict mode] which allows invalid input to be stored in the first place? I
cannot reproduce with the following test against MySQL with strict mode
enabled (which is the default). In strict mode the data is truncated as
expected

{{{#!diff
diff --git a/tests/lookup/test_decimalfield.py
b/tests/lookup/test_decimalfield.py
index d938ccf649..d4adb859c7 100644
--- a/tests/lookup/test_decimalfield.py
+++ b/tests/lookup/test_decimalfield.py
@@ -35,3 +35,9 @@ def test_lt(self):
def test_lte(self):
qs = self.queryset.filter(qty_needed__lte=0)
self.assertCountEqual(qs, [self.p1, self.p2])
+
+ def test_eq_truncation(self):
+ from decimal import Decimal
+
+ obj = Product.objects.create(name="Product1",
qty_target=Decimal("2000.003"))
+
self.assertEqual(Product.objects.get(qty_target=Decimal("2000.00")), obj)
}}}

Note that the test do fail when MySQL strict mode is disabled and as
expected on SQLite [https://www.sqlite.org/datatypes.html where pretty
much any data type is stored as text].

I let others chime in but to me this is a case of ''invalid'' or
''wontfix''
--
Ticket URL: <https://code.djangoproject.com/ticket/36574#comment:3>

[Django]

#36574: Regression: DecimalField values are no longer quantized before written to
the DB
-------------------------------------+-------------------------------------
Reporter: Aaron Mader | Owner: (none)

Type: Bug | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Normal | Resolution: invalid
Keywords: decimal_places | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* keywords: => decimal_places
* resolution: => invalid
* status: new => closed

Comment:

Agree with the above assessments.

The [https://docs.djangoproject.com/en/5.2/releases/5.2/#database-backend-
api release note] describes this bugfix as relevant only to third-party
database backends, so although we don't normally document bugfixes, I
could be open to an addition to indicate that silent truncation no longer
happens when values are prepped for insertion, since we have this report
of a user relying on it.
--
Ticket URL: <https://code.djangoproject.com/ticket/36574#comment:4>