[Django] #36855: Field.pre_save() called twice during save() in Django 6.0 when inserting new records

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
-------------------------------------+-------------------------------------
Reporter: nino-robotfutures- | Type: Bug
co | Component: Database
Status: new | layer (models, ORM)
Version: 6.0 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Commit 94680437 introduced a change where Field.pre_save() can be called
twice during Model.save() for new records:

Diff:
https://github.com/django/django/commit/94680437a45a71c70ca8bd2e68b72aa1e2eff337
Full method:
https://github.com/django/django/blob/94680437a45a71c70ca8bd2e68b72aa1e2eff337/django/db/models/base.py#L1085-L1164

Previously, pre_save() was only called once in the values comprehension.

Impact:

This is a breaking change for custom model fields that:
- Have side effects in pre_save() (e.g., registering
transaction.on_commit callbacks)
- Transform the value in a way that's not idempotent
- Maintain internal state based on pre_save() being called once per save

Example:

A custom field that registers an on_commit callback in pre_save() will
now register it twice, causing the callback to execute twice.

Expected behavior:

Field.pre_save() should be called exactly once per Model.save()
operation, regardless of whether the save results in an INSERT or UPDATE.
--
Ticket URL: <https://code.djangoproject.com/ticket/36855>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records

---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)
Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+------------------------------------
Changes (by Jacob Walls):

* component: Database layer (models, ORM) => Documentation
* keywords: => pre_save
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thanks for the report. This was discussed just a couple days ago, but I'm
happy to have a new ticket for clarity.

I suggested documenting that `pre_save()` functions should be idempotent
in ticket:36847#comment:11, but Simon pointed out we should update our own
`DateField.pre_save` implementations first. (They aren't technically
idempotent, but they're pretty close. They're certainly side-effect free.)

Some of the use cases you mention seem a better fit for the `pre_save`
signal than the `pre_save()` field method, but I'd have to look closer.
Committing to support for non-idempotent `pre_save()` methods is a new
feature that would merit some discussion elsewhere on the forum or django
/new-features. Take the example of another field method:
`ArrayField.get_db_prep_value` calls `get_db_prep_value()` on each of its
base fields. For that reason if `get_db_prep_value()` had side effects,
that would be a problem.

We likely should have documented this as a minor incompatible change in
6.0, so I'm reframing this as a documentation fix for the release notes
for now. Django has a grain, and one way to follow the grain is ensure
that `pre_save()` only preprocesses single values without side effects.
Side effects can be orchestrated in many other ways.
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:1>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records

--------------------------------------+------------------------------------
Reporter: nino-robotfutures-co | Owner: (none)

Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Jacob Walls):

* cc: Simon Charette (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:2>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records

---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)

Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+------------------------------------
Comment (by Nino Walker):

Thank you for the prompt reply. The specific context was with the django-
gcp library, which I was testing for compatibility with 6.0. I haven't
been in the Django internals since 1.2, so I won't argue the virtues of
your position. Below is my exact context workaround:

https://github.com/ninowalker/django-gcp/pull/2/changes

Specific workaround: https://github.com/ninowalker/django-
gcp/pull/2/changes#diff-
d92dddc7dd622b9917b9a6f5b0a4dee29f5c8d5c1870831953399c681626d9ac

Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:3>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)
Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Changes (by HarshAlreja):

* has_patch: 0 => 1

Comment:

PR created: https://github.com/django/django/pull/20520
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:4>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)
Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Changes (by Jacob Walls):

* has_patch: 1 => 0

Comment:

PR didn't target release notes.
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:5>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)
Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Comment (by Kundan Yadav):

hey can i work on it ?
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:6>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+------------------------------------
Reporter: Nino Walker | Owner: (none)
Type: Bug | Status: new
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Comment (by Jacob Walls):

What we need is a single sentence added under the ''Miscellaneous''
section under ''Backwards incompatible changes in 6.0'' in 6.0.txt. If
that sounds appealing, go right ahead.
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:7>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records

---------------------------------+----------------------------------------
Reporter: Nino Walker | Owner: Kundan Yadav
Type: Bug | Status: assigned

Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+----------------------------------------
Changes (by Kundan Yadav):

* owner: (none) => Kundan Yadav
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:8>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+----------------------------------------
Reporter: Nino Walker | Owner: Kundan Yadav
Type: Bug | Status: assigned
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+----------------------------------------
Changes (by Kundan Yadav):

* has_patch: 0 => 1

Comment:

https://github.com/django/django/pull/20534
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:9>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
---------------------------------+----------------------------------------
Reporter: Nino Walker | Owner: Kundan Yadav
Type: Bug | Status: assigned
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:
Keywords: pre_save | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
---------------------------------+----------------------------------------

Changes (by Jacob Walls):

* needs_better_patch: 0 => 1

Comment:

The wrong file was edited.
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:10>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records

-------------------------------------+-------------------------------------

Reporter: Nino Walker | Owner: Kundan
| Yadav
Type: Bug | Status: assigned
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution:

Keywords: pre_save | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:11>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
-------------------------------------+-------------------------------------
Reporter: Nino Walker | Owner: Kundan
| Yadav

Type: Bug | Status: closed
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution: fixed

Keywords: pre_save | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls <jacobtylerwalls@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"924156072ecf61ef9cf50fa0d4d553a4c0d416c2" 92415607]:
{{{#!CommitTicketReference repository=""
revision="924156072ecf61ef9cf50fa0d4d553a4c0d416c2"
Fixed #36855, Refs #27222 -- Mentioned multiple invocations of
Field.pre_save() in 6.0 release notes.

Co-authored-by: Jacob Walls <jacobty...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:12>

[Django]

#36855: Field.pre_save() called twice during save() in Django 6.0 when inserting
new records
-------------------------------------+-------------------------------------
Reporter: Nino Walker | Owner: Kundan
| Yadav
Type: Bug | Status: closed
Component: Documentation | Version: 6.0
Severity: Release blocker | Resolution: fixed
Keywords: pre_save | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls <jacobtylerwalls@…>):

In [changeset:"8027c0a8d2584a519d32a7f64d8a8c928f702021" 8027c0a]:
{{{#!CommitTicketReference repository=""
revision="8027c0a8d2584a519d32a7f64d8a8c928f702021"
[6.0.x] Fixed #36855, Refs #27222 -- Mentioned multiple invocations of

Field.pre_save() in 6.0 release notes.

Co-authored-by: Jacob Walls <jacobty...@gmail.com>

Backport of 924156072ecf61ef9cf50fa0d4d553a4c0d416c2 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36855#comment:13>