Re: [Django] #36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of SQL NULL

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* severity: Normal => Release blocker

Comment:

#36440 was a dupe. Adam bisected it to a 5.2 commit, so elevating to
blocker for now.
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Adam Johnson):

* cc: Adam Johnson (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:5>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned

Component: Database layer | Version: 5.2

(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* version: dev => 5.2

--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:6>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* severity: Release blocker => Normal

Comment:

For what is worth, the test_json_field_json_null_value as presented in the
proposed branch fails in 5.1.x, 4.2.x, 4.1.x, 4.0.x and 3.2.x, so unclear
this is truly a release blocker for 5.2.x.

Adam, could you provide more details about your bisect outcome?
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:7>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Clifford Gama):

* severity: Normal => Release blocker

Comment:

Hello Natalia. I did a git bisect and the issue is indeed a regression in
00c690efbc0b10f67924687f24a7b30397bf47d9. The test by
`test_json_field_json_null_value` is something I had assumed to have
always worked and isn't actually a regression test for the issue here.
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:8>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Johnson):

Adam, could you provide more details about your bisect outcome?

I bisected for my client project failure, so unfortunately, I can't really
share that. I found that the client test failed for
e306687a3a5507d59365ba9bf545010e5fd4b2a8.

I then minimized the issue into the test provided in
https://github.com/django/django/pull/19527 , which reproduces the
exception I encountered. I didn't bisect with that test before.

I just tried bisecting with it and found
1df0f998ae8d1626adaa9b807a43e4cbdfc590e2, the 5.2 backport of
00c690efbc0b10f67924687f24a7b30397bf47d9, is indeed the first commit that
makes my test fail.

So it seems we're working on the same issue, but it may be that there may
be some other failure lurking in the client project test. I'll try to
check, but no promises!
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:9>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: assigned
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:10>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama

Type: Bug | Status: closed

Component: Database layer | Version: 5.2
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"c1fa3fdd040718356e5a3b9a0fe699d73f47a940" c1fa3fdd]:
{{{#!CommitTicketReference repository=""
revision="c1fa3fdd040718356e5a3b9a0fe699d73f47a940"
Fixed #36419 -- Ensured for_save was propagated when resolving
expressions.

The for_save flag wasn't properly propagated when resolving expressions,
which
prevented get_db_prep_save() from being called in some cases. This
affected
fields like JSONField where None would be saved as JSON null instead of
SQL NULL.

Regression in 00c690efbc0b10f67924687f24a7b30397bf47d9.

Thanks to David Sanders and Simon Charette for reviews.

Co-authored-by: Adam Johnson <m...@adamj.eu>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:11>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"6fc620b4a8e91839b93af2b52d80bdbd5f8a1fcc" 6fc620b4]:
{{{#!CommitTicketReference repository=""
revision="6fc620b4a8e91839b93af2b52d80bdbd5f8a1fcc"
[5.2.x] Fixed #36419 -- Ensured for_save was propagated when resolving

expressions.

The for_save flag wasn't properly propagated when resolving expressions,
which
prevented get_db_prep_save() from being called in some cases. This
affected
fields like JSONField where None would be saved as JSON null instead of
SQL NULL.

Regression in 00c690efbc0b10f67924687f24a7b30397bf47d9.

Thanks to David Sanders and Simon Charette for reviews.

Co-authored-by: Adam Johnson <m...@adamj.eu>

Backport of c1fa3fdd040718356e5a3b9a0fe699d73f47a940 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:12>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"f5441e42da691ee2e7aeeb9be70f98e2bce6d17d" f5441e42]:
{{{#!CommitTicketReference repository=""
revision="f5441e42da691ee2e7aeeb9be70f98e2bce6d17d"
Refs #36419 -- Fixed BulkUpdateTests.test_json_field_sql_null() crash on
Oracle.

Follow up to c1fa3fdd040718356e5a3b9a0fe699d73f47a940.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:13>

[Django]

#36419: bulk_update() incorrectly updates JSONField None to JSON 'null' instead of
SQL NULL
-------------------------------------+-------------------------------------
Reporter: Clifford Gama | Owner: Clifford
| Gama
Type: Bug | Status: closed
Component: Database layer | Version: 5.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"aec11dbb4c2c66af8d09dc194a46950751b0ea34" aec11db]:
{{{#!CommitTicketReference repository=""
revision="aec11dbb4c2c66af8d09dc194a46950751b0ea34"
[5.2.x] Refs #36419 -- Fixed BulkUpdateTests.test_json_field_sql_null()

crash on Oracle.

Follow up to c1fa3fdd040718356e5a3b9a0fe699d73f47a940.

Backport of f5441e42da691ee2e7aeeb9be70f98e2bce6d17d from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36419#comment:14>