[Django] #35350: Crash on write of new instance when using GeneratedField

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt | Owner: nobody
Hegarty |
Type: Bug | Status: new
Component: Database | Version: 5.0
layer (models, ORM) |
Severity: Normal | Keywords: GeneratedField
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
We have an [https://github.com/django-import-export/django-import-
export/issues/1779 issue in django-import-export] where imports of new
instances fails when the model has a GeneratedField.

This occurs when an instance is being created which doesn't already exist
in the database, which the
[https://docs.djangoproject.com/en/5.0/ref/models/fields/#generatedfield
docs] imply is supported by GeneratedField:

A stored generated column is computed when it is written (inserted or
updated)

The stack trace is attached.

Steps to reproduce:

1. Clone [https://github.com/matthewhegarty/django-import-
export/tree/issue-1779-generatedfield this repo] and use the
'issue-1779-generatedfield' branch.
2. Create a [https://django-import-
export.readthedocs.io/en/4.0.0-rc.1/contributing.html#development venv]
3. Install the example app following the instructions [https://django-
import-export.readthedocs.io/en/4.0.0-rc.1/installation.html#example-app
here]
4. Navigate to "Books", click "Import", then select the attached
sample.csv file.
5. Click "Submit" and you will see the error.
--
Ticket URL: <https://code.djangoproject.com/ticket/35350>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------

Reporter: Matt Hegarty | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: GeneratedField | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Matt Hegarty):

* Attachment "sample.csv" added.

sample csv for import

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: GeneratedField | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Matt Hegarty):

* Attachment "st1.txt" added.

stack trace

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: GeneratedField | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Matt Hegarty:

Old description:

> We have an [https://github.com/django-import-export/django-import-
> export/issues/1779 issue in django-import-export] where imports of new
> instances fails when the model has a GeneratedField.
>
> This occurs when an instance is being created which doesn't already exist
> in the database, which the
> [https://docs.djangoproject.com/en/5.0/ref/models/fields/#generatedfield
> docs] imply is supported by GeneratedField:
>
> A stored generated column is computed when it is written (inserted or
> updated)
>
> The stack trace is attached.
>
> Steps to reproduce:
>
> 1. Clone [https://github.com/matthewhegarty/django-import-
> export/tree/issue-1779-generatedfield this repo] and use the
> 'issue-1779-generatedfield' branch.
> 2. Create a [https://django-import-
> export.readthedocs.io/en/4.0.0-rc.1/contributing.html#development venv]
> 3. Install the example app following the instructions [https://django-
> import-export.readthedocs.io/en/4.0.0-rc.1/installation.html#example-app
> here]
> 4. Navigate to "Books", click "Import", then select the attached
> sample.csv file.
> 5. Click "Submit" and you will see the error.

New description:

We have an [https://github.com/django-import-export/django-import-
export/issues/1779 issue in django-import-export] where imports of new
instances fails when the model has a GeneratedField.

This occurs when an instance is being created which doesn't already exist
in the database, which the
[https://docs.djangoproject.com/en/5.0/ref/models/fields/#generatedfield
docs] imply is supported by GeneratedField:

A stored generated column is computed when it is written (inserted or
updated)

The stack trace is attached.

Steps to reproduce:

1. Clone [https://github.com/matthewhegarty/django-import-
export/tree/issue-1779-generatedfield this repo] and use the
'issue-1779-generatedfield' branch.
2. Create a [https://django-import-
export.readthedocs.io/en/4.0.0-rc.1/contributing.html#development venv]
3. Install the example app following the instructions [https://django-
import-export.readthedocs.io/en/4.0.0-rc.1/installation.html#example-app
here]
4. Navigate to "Books", click "Import", then select the attached
sample.csv file.
5. Click "Submit" and you will see the error.

* python 3.12
* django 5.0.3

--
--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:1>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: GeneratedField | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

This is similar to #35019 (fixed by
03af8fbd0f1cfc08edadda248cf5d3498e8ed2f7) but when dealing with instances
that have a primary key assigned and `force_insert` is not specified.

When this is the case `Mode.save`
[https://github.com/django/django/blob/4636baec179d8733e92c1eccfa669bd72d739735/django/db/models/base.py#L1094-L1112
will first attempt to perform an] `UPDATE` as it cannot know whether or
not the row exists.

It seems that the `non_pk` logic in `_save_table` needs to be adapted to
also needs to exclude deferred fields.
--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:2>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 5.0
(models, ORM) |

Severity: Release blocker | Resolution:
Keywords: GeneratedField | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:3>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------

Reporter: Matt Hegarty | Owner: Sarah
| Boyce
Type: Bug | Status: assigned

Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: GeneratedField | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* has_patch: 0 => 1
* owner: nobody => Sarah Boyce
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:4>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Sarah
| Boyce
Type: Bug | Status: assigned
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: GeneratedField | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:5>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Sarah
| Boyce

Type: Bug | Status: closed

Component: Database layer | Version: 5.0
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: GeneratedField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"8b53560eea9f10a1271d3bdf765dc6f969c7d9d5" 8b53560e]:
{{{#!CommitTicketReference repository=""
revision="8b53560eea9f10a1271d3bdf765dc6f969c7d9d5"
Fixed #35350 -- Fixed save() with pk set on models with GeneratedFields.

Thanks Matt Hegarty for the report and Simon Charette and Natalia Bidart
for the reviews.

Regression in f333e35.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:6>

[Django]

#35350: Crash on write of new instance when using GeneratedField
-------------------------------------+-------------------------------------
Reporter: Matt Hegarty | Owner: Sarah
| Boyce
Type: Bug | Status: closed
Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: GeneratedField | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Sarah Boyce <42296566+sarahboyce@…>):

In [changeset:"f075a19e856ced33ddfb0e1330b5c277ddb14bfe" f075a19e]:
{{{#!CommitTicketReference repository=""
revision="f075a19e856ced33ddfb0e1330b5c277ddb14bfe"
[5.0.x] Fixed #35350 -- Fixed save() with pk set on models with

GeneratedFields.

Thanks Matt Hegarty for the report and Simon Charette and Natalia Bidart
for the reviews.

Regression in f333e35.

Backport of 8b53560eea9f10a1271d3bdf765dc6f969c7d9d5 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35350#comment:7>