[Django] #35422: Migration crash when renaming a field referenced in GeneratedField.expression on SQLite

[Django]

#35422: Migration crash when renaming a field referenced in
GeneratedField.expression on SQLite
---------------------------------------+------------------------
Reporter: Sarah Boyce | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 5.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------+------------------------
Given a model (migrated) on SQLite
{{{
class Foo(models.Model):
name = models.CharField(max_length=10)
lower_name = models.GeneratedField(
expression=Lower("name"),
output_field=models.CharField(max_length=10),
db_persist=True,
)
}}}
when you update to
{{{
class Foo(models.Model):
surname = models.CharField(max_length=10)
lower_name = models.GeneratedField(
expression=Lower("surname"),
output_field=models.CharField(max_length=10),
db_persist=True,
)
}}}
and makemigrations, you then get asked `Was foo.name renamed to
foo.surname (a CharField)? [y/N]` and say `y` (because it was)
You then get a crash when migrating:


{{{
Operations to perform:
Apply all migrations: app3
Running migrations:
Applying
app3.0005_rename_name_foo_surname_alter_foo_lower_name...Traceback (most
recent call last):
File "path_to_project\mysite\manage.py", line 22, in <module>
main()
File "path_to_project\mysite\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "path_to_project\venv\Lib\site-
packages\django\core\management\__init__.py", line 442, in
execute_from_command_line
utility.execute()
File "path_to_project\venv\Lib\site-
packages\django\core\management\__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "path_to_project\venv\Lib\site-
packages\django\core\management\base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "path_to_project\venv\Lib\site-
packages\django\core\management\base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\core\management\base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\core\management\commands\migrate.py", line 356, in handle
post_migrate_state = executor.migrate(
^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\migrations\executor.py", line 135, in migrate
state = self._migrate_all_forwards(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\migrations\executor.py", line 167, in
_migrate_all_forwards
state = self.apply_migration(
^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\migrations\executor.py", line 255, in apply_migration
state = migration.apply(state, schema_editor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\migrations\migration.py", line 132, in apply
operation.database_forwards(
File "path_to_project\venv\Lib\site-
packages\django\db\migrations\operations\fields.py", line 241, in
database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "path_to_project\venv\Lib\site-
packages\django\db\backends\base\schema.py", line 875, in alter_field
or old_field.generated_sql(self.connection)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\fields\generated.py", line 58, in generated_sql
resolved_expression = self.expression.resolve_expression(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\expressions.py", line 1052, in
resolve_expression
c.source_expressions[pos] = arg.resolve_expression(
^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\expressions.py", line 874, in resolve_expression
return query.resolve_ref(self.name, allow_joins, reuse, summarize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\sql\query.py", line 2010, in resolve_ref
join_info = self.setup_joins(
^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\sql\query.py", line 1863, in setup_joins
path, final_field, targets, rest = self.names_to_path(
^^^^^^^^^^^^^^^^^^^
File "path_to_project\venv\Lib\site-
packages\django\db\models\sql\query.py", line 1768, in names_to_path
raise FieldError(
django.core.exceptions.FieldError: Cannot resolve keyword 'name' into
field. Choices are: id, lower_name, surname
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35422>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35422: Migration crash when renaming a field referenced in
GeneratedField.expression on SQLite

---------------------------------+------------------------------------

Reporter: Sarah Boyce | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 5.0

Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

---------------------------------+------------------------------------
Changes (by Natalia Bidart):

* cc: Simon Charette, Mariusz Felisiak, Lily Foote (added)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thank you Sarah, I reproduced and confirmed that this is not a dupe of
#35359 (or at least the fix for that ticket is not fixing this issue).

This is a release blocker for 5.0 given that `GeneratedField` was added in
that Django version. I also reproduced in
f333e3513e8bdf5ffeb6eeb63021c230082e6f95.
--
Ticket URL: <https://code.djangoproject.com/ticket/35422#comment:1>

[Django]

#35422: Migration crash when renaming a field referenced in
GeneratedField.expression on SQLite
---------------------------------+------------------------------------
Reporter: Sarah Boyce | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------
Changes (by Natalia Bidart):

* cc: David Sanders, Bhuvnesh (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/35422#comment:2>

[Django]

#35422: Migration crash when renaming a field referenced in
GeneratedField.expression on SQLite
---------------------------------+------------------------------------
Reporter: Sarah Boyce | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 5.0
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Comment (by Simon Charette):

I don't think the problem is specific to SQLite.

The non-invasive back portable solution I can think of is for `FieldError`
to be caught and also result in a `ValueError` pointing out that the
generated field must be removed first.
--
Ticket URL: <https://code.djangoproject.com/ticket/35422#comment:3>