[Django] #29000: RenameModel does not rename M2M column when run after AlterField/RenameField
Django
AlterField/RenameField
-----------------------------------------------+------------------------
Reporter: David Nelson Adamec | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------------+------------------------
Encountered this on a project at work. I have some models like so:
{{{#!python
class ModelA(models.Model):
name_renamed = models.CharField(max_length=10)
class ModelBRenamed(models.Model):
a = models.ForeignKey(ModelA, null=True)
class ModelC(models.Model):
b_set = models.ManyToManyField(ModelBRenamed)
}}}
I have a migration 0002_renamefield to rename a field on ModelA:
{{{#!python
class Migration(migrations.Migration):
dependencies = [
('myapp', '0001_initial'),
]
operations = [
migrations.RenameField(
model_name='modela',
old_name='name',
new_name='name_renamed',
),
]
}}}
Then a migration 0003_renamemodel to rename ModelB:
{{{#!python
class Migration(migrations.Migration):
dependencies = [
('myapp', '0002_renamefield'),
]
operations = [
migrations.RenameModel(
old_name='ModelB',
new_name='ModelBRenamed',
),
]
}}}
If the two migrations are run together, then the `modelb_id` column in
`myapp_modelc_b_set` will not be renamed to `modelbrenamed_id`. However,
if I run the migrations one at a time, the column will be renamed as
expected.
I think this is related to #27737. When ModelA is reloaded in the
RenameField migration, ModelB is reloaded due to it's foreign key but is
missing its relationship to ModelC. When the RenameModel migration is run,
ModelC is not included in ModelB's related_objects, so the through table's
column is not renamed.
I've reproduced this using Django 1.11 on either MySQL or SQLite.
--
Ticket URL: <https://code.djangoproject.com/ticket/29000>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: nobody
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* type: Uncategorized => Bug
* component: Uncategorized => Migrations
Comment:
Please test with Django 2.0 as there are changes there which may resolve
this.
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:1>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: nobody
Component: Migrations | Version: 1.11
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by David Nelson Adamec):
Sorry, should have just done that from the start. Tested in Django 2.0.1,
and the issue still occurs.
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:2>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: nobody
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by David Nelson Adamec):
* version: 1.11 => 2.0
Old description:
New description:
{{{#!python
class Migration(migrations.Migration):
{{{#!python
class Migration(migrations.Migration):
RenameField migration, ModelB is reloaded due to its foreign key but is
missing its relationship to ModelC. When the RenameModel migration is run,
ModelC is not included in ModelB's related_objects, so the through table's
column is not renamed.
I've reproduced this using Django 1.11 on either MySQL or SQLite.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:3>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: nobody
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Unreviewed => Accepted
Comment:
Thanks, good bug report. I can also reproduce on PostgreSQL with Django
master (e2908ecb3e20a629be801ddb826c474f7e7023ea).
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:4>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner:
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by shangdahao):
* owner: nobody => shangdahao
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:5>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner:
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by David Nelson Adamec):
After doing a little more digging, it appears that the `reload_model` with
`delay=True` in the RenameField migration is the culprit. Because of the
`delay` flag, only the original model and its directly related models
(ModelA and ModelB in the example) are reloaded, so ModelC is untouched
and its `b_set` field continues to point at the old version of ModelB
that's no longer registered. Since ModelC isn't pointing at the new
ModelB, it's not included in ModelB's `related_objects`.
I'm not sure what the performance implication would be if delayed loading
were removed from RenameField/AlterField, but I'm sure it's not good.
Perhaps there's another clever solution out there, but I feel a bit out of
my depth. Here's the workaround I'm using for the time being.
{{{#!python
class RenameModelWorkaround(migrations.RenameModel):
def database_forwards(self, app_label, schema_editor, from_state,
to_state):
from_state.reload_model(app_label, self.old_name_lower)
super(RenameModelWorkaround, self).database_forwards(
app_label, schema_editor, from_state, to_state)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:6>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: (none)
Type: Bug | Status: new
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by shangdahao):
* status: assigned => new
* owner: shangdahao => (none)
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:7>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Jeff):
* owner: (none) => Jeff
* status: new => assigned
Comment:
Unsurprisingly, this bug also breaks backwards migrations, as the names in
the bridge table don't match expected values.
For the models/migrations in the tickets description, backwards migrations
produces the expected result:
{{{
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/__init__.py",
line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/__init__.py",
line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/base.py",
line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/base.py",
line 331, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/base.py",
line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/core/management/commands/migrate.py",
line 203, in handle
fake_initial=fake_initial,
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/executor.py",
line 121, in migrate
state = self._migrate_all_backwards(plan, full_plan, fake=fake)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/executor.py",
line 196, in _migrate_all_backwards
self.unapply_migration(states[migration], migration, fake=fake)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/executor.py",
line 262, in unapply_migration
state = migration.unapply(state, schema_editor)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/migration.py",
line 175, in unapply
operation.database_backwards(self.app_label, schema_editor,
from_state, to_state)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/operations/models.py",
line 377, in database_backwards
self.database_forwards(app_label, schema_editor, from_state, to_state)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/migrations/operations/models.py",
line 349, in database_forwards
to_field,
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/base/schema.py",
line 508, in alter_field
return self._alter_many_to_many(model, old_field, new_field, strict)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/base/schema.py",
line 863, in _alter_many_to_many
new_field.remote_field.through._meta.get_field(new_field.m2m_reverse_field_name()),
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/base/schema.py",
line 523, in alter_field
old_db_params, new_db_params, strict)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/base/schema.py",
line 605, in _alter_field
self.execute(self._rename_field_sql(model._meta.db_table, old_field,
new_field, new_type))
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/base/schema.py",
line 133, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/utils.py",
line 100, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/utils.py",
line 68, in execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/utils.py",
line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/utils.py",
line 85, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/utils.py", line
89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/utils.py",
line 85, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/dist-
packages/Django-2.1.dev20180514000620-py3.6.egg/django/db/backends/mysql/base.py",
line 71, in execute
return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 250, in
execute
self.errorhandler(self, exc, value)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 50,
in defaulterrorhandler
raise errorvalue
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 247, in
execute
res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 411, in
_query
rowcount = self._do_query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 374, in
_do_query
db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 292,
in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1054, "Unknown column
'modelbrenamed_id' in 'temp_modelc_b_set'")
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:8>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Jeff):
Pull request is out: https://github.com/django/django/pull/9975
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:9>
Django
AlterField/RenameField
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Jeff):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: assigned
Component: Migrations | Version: 2.0
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: closed
Component: Migrations | Version: 2.0
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"fcc4e251dbc917118f73d7187ee2f4cbf3883f36" fcc4e251]:
{{{
#!CommitTicketReference repository=""
revision="fcc4e251dbc917118f73d7187ee2f4cbf3883f36"
Fixed #29000 -- Fixed RenameModel's renaming of a M2M column when run
after RenameField.
Regression in 45ded053b1f4320284aa5dac63052f6d1baefea9.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: closed
Component: Migrations | Version: 2.0
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"2b7b199390336f7068ef8cb8d67b2e83fb6b818e" 2b7b1993]:
{{{
#!CommitTicketReference repository=""
revision="2b7b199390336f7068ef8cb8d67b2e83fb6b818e"
[2.1.x] Fixed #29000 -- Fixed RenameModel's renaming of a M2M column when
run after RenameField.
Regression in 45ded053b1f4320284aa5dac63052f6d1baefea9.
Backport of fcc4e251dbc917118f73d7187ee2f4cbf3883f36 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: David Nelson Adamec | Owner: Jeff
Type: Bug | Status: closed
Component: Migrations | Version: 2.0
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"ad811335bdd57d65dddbc05375ec3a70b59c96fb" ad81133]:
{{{
#!CommitTicketReference repository=""
revision="ad811335bdd57d65dddbc05375ec3a70b59c96fb"
Refs #29000 -- Restored delayed model rendering of RenameField.
Non-delayed rendering is unnecessary and wasteful now that state models
relationship consistency on delayed reload is ensured.
This partly reverts commit fcc4e251dbc917118f73d7187ee2f4cbf3883f36.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29000#comment:14>