Re: [Django] #35167: JSONField get_db_prep_value being called with `Cast` types

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by HeejunShin):

Could i try this ticket?
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by bcail):

@HeejunShin, nobody owns this ticket, so you're welcome to work on this
ticket. You can assign it to yourself and/or add comments as you work on
it.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:10>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------

Reporter: Samantha Hughes | Owner: Almaz
Type: Bug | Status: assigned

Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Almaz):

* owner: nobody => Almaz
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:11>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Almaz
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Almaz):

Is this a relevant problem?

Because, after some period of exploration, I see that most of the code
were updated.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:12>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------

Reporter: Samantha Hughes | Owner: (none)
Type: Bug | Status: new

Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Almaz):

* owner: Almaz => (none)
* status: assigned => new

--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:13>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HeejunShin):

Is this ticket solved?
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:14>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

Replying to [comment:14 HeejunShin]:
> Is this ticket solved?

No. While it has an inline tentative patch no one
[https://docs.djangoproject.com/en/5.0/internals/contributing/writing-code
/submitting-patches/#claiming-tickets has opened a PR on Github with the
proposed changes] including a regression test yet.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:15>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HeejunShin):

Replying to [comment:15 Simon Charette]:

> Replying to [comment:14 HeejunShin]:
> > Is this ticket solved?
>
> No. While it has an inline tentative patch no one
[https://docs.djangoproject.com/en/5.0/internals/contributing/writing-code
/submitting-patches/#claiming-tickets has opened a PR on Github with the
proposed changes] including a regression test yet.

> thank you for youre response.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:16>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by HeejunShin):

Thank you for youre response.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:17>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:

| HeejunShin
Type: Bug | Status: assigned

Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by HeejunShin):

* owner: (none) => HeejunShin

* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:18>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HeejunShin):

Replying to [comment:15 Simon Charette]:

thank you for youre response.

--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:19>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by HeejunShin):

Sorry for the slow progress. I am currently analyzing the code looking at
the customizing doc of the Field class.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:20>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia Bidart):

Ticket #35381 is related to this.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:21>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HeejunShin):

Replying to [comment:21 Natalia Bidart]:

> Ticket #35381 is related to this.

Is this a problem that needs to be resolved as quickly as possible?
Should I also refer to the ticket you mentioned to solve the problem?
Any advice would be greatly appreciated.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:22>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia Bidart):

Hello HeejunShin, thank you for asking! You can take the time you need to
work on this ticket, in general no ticket is urgent unless is a release
blocker, which is not the case for this one.
Also, if you can't work on this anymore, that is also fine and you can
just de-assign yourself from the ticket and eventually someone else would
pick it up.

Django is built and maintained by its community and everyone contributes
their time and energy when they can.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:23>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HeejunShin):

Replying to [comment:23 Natalia Bidart]:

> Hello HeejunShin, thank you for asking! You can take the time you need
to work on this ticket, in general no ticket is urgent unless is a release
blocker, which is not the case for this one.
> Also, if you can't work on this anymore, that is also fine and you can
just de-assign yourself from the ticket and eventually someone else would
pick it up.
>
> Django is built and maintained by its community and everyone contributes
their time and energy when they can.

thank you for your response.
I'll try my best.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:24>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by HeejunShin):

It's really late though
I was just analyzing the code and reproduced the bug.
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:25>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner:
| HeejunShin
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by HeejunShin):

i have a question.
maybe The bug reproduction seems to be incorrect.

i tested using 2 ways.

{{{
tests[1].json = {"test": Cast('good', output_field=CharField())}
Test.objects.bulk_update(tests_will_be_updated, fields=["json"])
}}}

{{{
tests[1].json = Cast("good", output_field=CharField())
Test.objects.bulk_update(tests_will_be_updated, fields=["json"])
}}}

first one, i got "raise TypeError(f'Object of type {o.__class__.__name__}
'TypeError: Object of type Cast is not JSON serializable" error.
but value argument type in get_db_prep_value is dict.

second one, i could not get error.
value argument type in get_db_prep_value is <class
'django.db.models.expressions.Case'>. (CASE WHEN <WhereNode: (AND:
Exact(Col(polls_test, polls.Test.id), 2))> THEN Cast(Col(polls_test,
polls.Test.json)), ELSE Value(None)).

i thought first one is type dict, so this is wrong.
is second one wrong way that reproduce bug case?
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:26>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------

Reporter: Samantha Hughes | Owner: Jacob
| Walls

Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: JSONField | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* has_patch: 0 => 1
* owner: HeejunShin => Jacob Walls

Comment:

[https://github.com/django/django/pull/19183 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:27>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:

Keywords: JSONField | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:28>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls

Type: Bug | Status: closed

Component: Database layer | Version: 4.2
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: JSONField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"0bf412111be686b6b23e00863f5d449d63557dbf" 0bf4121]:
{{{#!CommitTicketReference repository=""
revision="0bf412111be686b6b23e00863f5d449d63557dbf"
Fixed #35167 -- Delegated to super() in JSONField.get_db_prep_save().

Avoids reports of bulk_update() sending Cast expressions
to JSONField.get_prep_value().

Co-authored-by: Simon Charette <chare...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:29>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: JSONField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"9525135698bd4f97cf1431776ef52ae393dfb3c0" 9525135]:
{{{#!CommitTicketReference repository=""
revision="9525135698bd4f97cf1431776ef52ae393dfb3c0"
[5.2.x] Fixed #35167 -- Delegated to super() in

JSONField.get_db_prep_save().

Avoids reports of bulk_update() sending Cast expressions
to JSONField.get_prep_value().

Co-authored-by: Simon Charette <chare...@gmail.com>

Backport of 0bf412111be686b6b23e00863f5d449d63557dbf from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:30>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: JSONField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"7500044a825b3e6695f4f0f9e56d5bdc0c5d7988" 7500044]:
{{{#!CommitTicketReference repository=""
revision="7500044a825b3e6695f4f0f9e56d5bdc0c5d7988"
Refs #35167 -- Fixed test_bulk_update_custom_get_prep_value() crash on
databases that don't support primitives in JSONFields.

For example on Oracle < 21c.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:31>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: JSONField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"43aa6990ac0fa8472397a3c4e0f5d963006af798" 43aa6990]:
{{{#!CommitTicketReference repository=""
revision="43aa6990ac0fa8472397a3c4e0f5d963006af798"
[5.2.x] Refs #35167 -- Fixed test_bulk_update_custom_get_prep_value()

crash on databases that don't support primitives in JSONFields.

For example on Oracle < 21c.

Backport of 7500044a825b3e6695f4f0f9e56d5bdc0c5d7988 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:32>

[Django]

#35167: JSONField get_db_prep_value being called with `Cast` types
-------------------------------------+-------------------------------------
Reporter: Samantha Hughes | Owner: Jacob
| Walls
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: JSONField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

In [changeset:"77666f2fa1ef93f7b7728a565260229918d51532" 77666f2]:

Refs #35167 -- Removed hardcoded pk in
test_bulk_update_custom_get_prep_value().
--
Ticket URL: <https://code.djangoproject.com/ticket/35167#comment:33>