[Django] #35638: validate_constraints() fails on models with fields using db_default

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: dev | Severity: Normal
Keywords: db_default | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Follow up to #35223

Related forum thread: https://forum.djangoproject.com/t/proposal-to-have-
db-default-considered-in-model-forms/33358/3

Given the model

{{{
class Foo(models.Model):
bar = models.CharField(db_default="bar")

class Meta:
constraints = [
models.CheckConstraint(check=models.Q(bar="bar"),
name="is_bar"),
]
}}}

validating constraints should ideally not fail but currently does:

{{{
>>> foo = Foo()
>>> foo.validate_constraints()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/path/to/django/django/db/models/base.py", line 1598, in
validate_constraints
raise ValidationError(errors)
django.core.exceptions.ValidationError: {'__all__': ['Constraint “is_bar”
is violated.']}
}}}

I couldn't find an easy workaround for this.

If I apply this small patch to `Q.check()`, then `validate_constraints()`
uses the default expression and runs a suitable check query. Perhaps can
take this as a starting point and add something similar to Django?

{{{
index 1bf396723e..5380cc17d0 100644
--- a/django/db/models/query_utils.py
+++ b/django/db/models/query_utils.py
@@ -120,12 +120,16 @@ class Q(tree.Node):
"""
# Avoid circular imports.
from django.db.models import BooleanField, Value
+ from django.db.models.expressions import DatabaseDefault
from django.db.models.functions import Coalesce
from django.db.models.sql import Query
from django.db.models.sql.constants import SINGLE

query = Query(None)
for name, value in against.items():
+ # not ideal, value is wrapped in a Value by
Model._get_field_value_map()
+ if isinstance(getattr(value, "value"), DatabaseDefault):
+ value = value.field.db_default
if not hasattr(value, "resolve_expression"):
value = Value(value)
query.add_annotation(value, name, select=False)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------

Reporter: David Sanders | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thank you for your report!

We should adjust `_get_field_value_map` to take into account `db_default`
(and possibly rename to `_get_field_expr_map`) so all constraints are
adjusted appropriately. I suspect that other constraints as well as the
`unique=True` and `unique_together` validations that `validate_unique`
provides are also affected e.g.

{{{#!python
class Foo(models.Model):
bar = models.CharField(unique=True, db_default="bar")

Foo.objects.create()
Foo().validate_unique() # No ValidationError raised but would result in
an integrity error on `save`
}}}

''I haven't tested the above but I don't see any special handling of
`db_default` in `validate_unique`''

Marking as ''release blocker'' as `db_default` was introduced in 5.0.
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:1>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by David Sanders):

> We should adjust _get_field_value_map to take into account db_default

(and possibly rename to _get_field_expr_map) so all constraints are
adjusted appropriately.

👍 Yup was also thinking this so good to see some confirmation there

> I suspect that other constraints as well as the unique=True and
unique_together validations that validate_unique provides are also
affected e.g.

Ah nice, I didn't think of this. Whoever picks this test up will need to
write a test for that too ☝️

I was thinking of starting a PR tomorrow if I have time, but if someone
else want to pick it up before then - you're welcome to do so.
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:2>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by David Sanders):

* has_patch: 0 => 1
* needs_better_patch: 0 => 1
* needs_docs: 0 => 1
* needs_tests: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:3>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------

Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned

Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* owner: (none) => David Sanders
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:4>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned

Component: Database layer | Version: 5.0

(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Sarah Boyce):

* version: dev => 5.0

--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:5>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Sarah Boyce):

* needs_better_patch: 1 => 0
* needs_docs: 1 => 0
* needs_tests: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:6>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by David Sanders):

> Patch needs improvement: unset

Just need to verify whether we still want to keep the changes to
`_get_field_value_map()` … may no longer necessary with Simon's more
recent suggestion
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:7>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_better_patch: 0 => 1

Comment:

Replying to [comment:7 David Sanders]:

> Just need to verify whether we still want to keep the changes to
`_get_field_value_map()` … may no longer necessary with Simon's more
recent suggestion

Ah sorry! I'll let you unset when you're ready
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:8>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: db_default | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by David Sanders):

* needs_better_patch: 1 => 0

Comment:

Turns out we did need it… unsetting pending any further reviews & follow-
ups 😅
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:9>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: db_default | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:10>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders

Type: Bug | Status: closed

Component: Database layer | Version: 5.0
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"509763c79952cde02d9f5b584af4278bdbed77b2" 509763c7]:
{{{#!CommitTicketReference repository=""
revision="509763c79952cde02d9f5b584af4278bdbed77b2"
Fixed #35638 -- Updated validate_constraints to consider db_default.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:12>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders

Type: Bug | Status: assigned

Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"91a038754bb516d29cb79f0fed4025436b5c5346" 91a03875]:
{{{#!CommitTicketReference repository=""
revision="91a038754bb516d29cb79f0fed4025436b5c5346"
Refs #35638 -- Avoided wrapping expressions with Value in
_get_field_value_map() and renamed to _get_field_expression_map().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:11>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders

Type: Bug | Status: closed

Component: Database layer | Version: 5.0
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"78654a29b851751f347d51dbfe5ba8a996c012ba" 78654a2]:
{{{#!CommitTicketReference repository=""
revision="78654a29b851751f347d51dbfe5ba8a996c012ba"
[5.1.x] Refs #35638 -- Avoided wrapping expressions with Value in

_get_field_value_map() and renamed to _get_field_expression_map().

Backport of 91a038754bb516d29cb79f0fed4025436b5c5346 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:13>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: closed
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"aed4ffe1897c51c0dfd8689c0834ca4ca0a8a03e" aed4ffe1]:
{{{#!CommitTicketReference repository=""
revision="aed4ffe1897c51c0dfd8689c0834ca4ca0a8a03e"
[5.1.x] Fixed #35638 -- Updated validate_constraints to consider
db_default.

Backport of 509763c79952cde02d9f5b584af4278bdbed77b2 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:14>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: closed
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"e88ef6a27d125da95e530be20b879fb4f540f7f6" e88ef6a]:
{{{#!CommitTicketReference repository=""
revision="e88ef6a27d125da95e530be20b879fb4f540f7f6"
[5.0.x] Refs #35638 -- Avoided wrapping expressions with Value in

_get_field_value_map() and renamed to _get_field_expression_map().

Backport of 91a038754bb516d29cb79f0fed4025436b5c5346 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:15>

[Django]

#35638: validate_constraints() fails on models with fields using db_default
-------------------------------------+-------------------------------------
Reporter: David Sanders | Owner: David
| Sanders
Type: Bug | Status: closed
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: db_default | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"333cfab512689490ea8a6a95598a94d77816fa2d" 333cfab5]:
{{{#!CommitTicketReference repository=""
revision="333cfab512689490ea8a6a95598a94d77816fa2d"
[5.0.x] Fixed #35638 -- Updated validate_constraints to consider
db_default.

Backport of 509763c79952cde02d9f5b584af4278bdbed77b2 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35638#comment:16>