[Django] #33194: Sqlite AlterField migration on a table with a unique index fails

[Django]

#33194: Sqlite AlterField migration on a table with a unique index fails
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 4.0
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
In Django 4.0a1, this model:
{{{
class Tag(models.Model):
name = models.SlugField(help_text="The tag key.")
value = models.CharField(max_length=150, help_text="The tag value.")

class Meta:
ordering = ["name", "value"]
constraints = [
models.UniqueConstraint(
"name",
"value",
name="unique_name_value",
)
]

def __str__(self):
return f"{self.name}={self.value}"

}}}

with these migrations, using sqlite:

{{{

class Migration(migrations.Migration):

initial = True

dependencies = [
]

operations = [
migrations.CreateModel(
name='Tag',
fields=[
('id', models.BigAutoField(auto_created=True,
primary_key=True, serialize=False, verbose_name='ID')),
('name', models.SlugField(help_text='The tag key.')),
('value', models.CharField(help_text='The tag value.',
max_length=200)),
],
options={
'ordering': ['name', 'value'],
},
),
migrations.AddConstraint(
model_name='tag',
constraint=models.UniqueConstraint(django.db.models.expressions.F('name'),
django.db.models.expressions.F('value'), name='unique_name_value'),
),
]

class Migration(migrations.Migration):

dependencies = [
('myapp', '0001_initial'),
]

operations = [
migrations.AlterField(
model_name='tag',
name='value',
field=models.CharField(help_text='The tag value.',
max_length=150),
),
]

}}}

raises this error:

{{{
manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, myapp, sessions
Running migrations:
Applying myapp.0002_alter_tag_value...python-BaseException
Traceback (most recent call last):
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\sqlite3\base.py", line 416, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: the "." operator prohibited in index expressions

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\core\management\base.py", line 373, in run_from_argv
self.execute(*args, **cmd_options)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\core\management\base.py", line 417, in execute
output = self.handle(*args, **options)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\core\management\base.py", line 90, in wrapped
res = handle_func(*args, **kwargs)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\core\management\commands\migrate.py", line 253, in handle
post_migrate_state = executor.migrate(
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\migrations\executor.py", line 126, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake,
fake_initial=fake_initial)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\migrations\executor.py", line 156, in
_migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake,
fake_initial=fake_initial)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\migrations\executor.py", line 236, in apply_migration
state = migration.apply(state, schema_editor)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\migrations\migration.py", line 125, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\migrations\operations\fields.py", line 225, in
database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\sqlite3\schema.py", line 140, in alter_field
super().alter_field(model, old_field, new_field, strict=strict)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\base\schema.py", line 618, in alter_field
self._alter_field(model, old_field, new_field, old_type, new_type,
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\sqlite3\schema.py", line 362, in _alter_field
self._remake_table(model, alter_field=(old_field, new_field))
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\sqlite3\schema.py", line 303, in _remake_table
self.execute(sql)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\base\schema.py", line 151, in execute
cursor.execute(sql, params)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 98, in execute
return super().execute(sql, params)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "D:\Projects\Development\sqliteerror\.venv\lib\site-
packages\django\db\backends\sqlite3\base.py", line 416, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: the "." operator prohibited in index
expressions
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33194>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.

-------------------------------------+-------------------------------------
Reporter: Mark | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* cc: Hannes Ljungberg (added)
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted

Comment:

Thanks for the report.

Regression in 3aa545281e0c0f9fac93753e3769df9e0334dbaa.

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:1>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------

Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: assigned

Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Hannes Ljungberg):

* owner: nobody => Hannes Ljungberg
* status: new => assigned

Comment:

Thanks for the report! Looks like we don't check if an `alias` is set on
the `Col` before we update it to `new_table` in
`Expressions.rename_table_references` when running `_remake_table`.

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:2>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Hannes Ljungberg):

PR: https://github.com/django/django/pull/14997

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:3>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:4>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: assigned

Component: Database layer | Version: 3.2

(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* version: 4.0 => 3.2

Comment:

> Not really sure if we need a backport to 3.2 but feels like this would
create issues for non-unique functional indexes on SQLite as well.

Good catch, it's a bug in 83fcfc9ec8610540948815e127101f1206562ead.

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:5>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: assigned
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:6>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg

Type: Bug | Status: closed

Component: Database layer | Version: 3.2
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* status: assigned => closed
* resolution: => fixed

Comment:

In [changeset:"86971c40909430a798e4e55b140004c4b1fb02ff" 86971c4]:
{{{
#!CommitTicketReference repository=""
revision="86971c40909430a798e4e55b140004c4b1fb02ff"
Fixed #33194 -- Fixed migrations when altering a field with functional
indexes/unique constraints on SQLite.

This adjusts Expressions.rename_table_references() to only update alias
when needed.

Regression in 83fcfc9ec8610540948815e127101f1206562ead.

Co-authored-by: Simon Charette <char...@users.noreply.github.com>
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:7>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"00aa3e0b9b8fac8fa5adab47249790d2ee2f767e" 00aa3e0b]:
{{{
#!CommitTicketReference repository=""
revision="00aa3e0b9b8fac8fa5adab47249790d2ee2f767e"
[4.0.x] Fixed #33194 -- Fixed migrations when altering a field with

functional indexes/unique constraints on SQLite.

This adjusts Expressions.rename_table_references() to only update alias
when needed.

Regression in 83fcfc9ec8610540948815e127101f1206562ead.

Co-authored-by: Simon Charette <char...@users.noreply.github.com>

Backport of 86971c40909430a798e4e55b140004c4b1fb02ff from main
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:8>

[Django]

#33194: Remaking table with unique constraint crashes on SQLite.
-------------------------------------+-------------------------------------
Reporter: Mark | Owner: Hannes
| Ljungberg
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"f5802a21c401b92764a9f3e2886144f3c5d77573" f5802a21]:
{{{
#!CommitTicketReference repository=""
revision="f5802a21c401b92764a9f3e2886144f3c5d77573"
[3.2.x] Fixed #33194 -- Fixed migrations when altering a field with
functional indexes on SQLite.

This adjusts Expressions.rename_table_references() to only update alias
when needed.

Regression in 83fcfc9ec8610540948815e127101f1206562ead.

Co-authored-by: Simon Charette <char...@users.noreply.github.com>

Backport of 86971c40909430a798e4e55b140004c4b1fb02ff from main.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33194#comment:9>