[Django] #31329: Converting from concrete to abstract inheritance creates invalid migrations
Django
--------------------------------------------+------------------------
Reporter: Stephen Finucane | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------------+------------------------
I currently have two models using concrete inheritance and would like to
split these out into wholly separate tables, dropping the base table in
the process. To do this, I have copied all columns from the base table
into the "child" tables, and would now like to drop the remaining
reference to the base table. To do this, I was hoping to (a) add a new
auto-incrementing `id` column to the child tables that is not yet a
primary key, (b) use a data migration to copy the data from the
`parent_ptr_id` column to the `id` column, and (c) drop the
`parent_ptr_id` column and switch `id` to be the primary key. However,
I've noticed that regardless of what I do, I won't be able to run the
migration or generate new migrations because of errors like the below.
{{{
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/__init__.py", line 401, in
execute_from_command_line
utility.execute()
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/core/management/commands/makemigrations.py", line 168, in
handle
migration_name=self.migration_name,
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/autodetector.py", line 43, in changes
changes = self._detect_changes(convert_apps, graph)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/autodetector.py", line 128, in
_detect_changes
self.old_apps = self.from_state.concrete_apps
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/state.py", line 213, in concrete_apps
self.apps = StateApps(self.real_apps, self.models,
ignore_swappable=True)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/state.py", line 272, in __init__
self.render_multiple([*models.values(), *self.real_models])
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/state.py", line 307, in render_multiple
model.render(self)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/migrations/state.py", line 578, in render
return type(self.name, bases, body)
File "/tmp/django-error/.venv/lib/python3.7/site-
packages/django/db/models/base.py", line 229, in __new__
base.__name__,
django.core.exceptions.FieldError: Local field 'id' in class 'Child'
clashes with field of the same name from base class 'Parent'.
}}}
I suspect this is because the state tracking is not accurately capturing
the change in table bases, but I don't know how to override this without
faking dropping the entire table and recreating it. I was hoping for an
`AlterModelBases` API but there doesn't appear to be one and I'm not sure
where to start adding one.
Note that I'm taking this approach because the actual model mapping to
`Child` holds a lot of data, and I suspect creating a new `Child2` model,
copying everything across, and then dropping `Parent` and `Child`, would
likely take many hours if not days to do. I would drop down to raw SQL for
this but I've yet to figure out how to correctly update the foreign key
constraints such that Django can keep track of them.
Minimal reproducer below.
== Steps to reproduce
Create a new project and app. In the app, create the following
`models.py`.
{{{#!python
from django.db import models
class Parent(models.Model):
pass
class Child(Parent):
name = models.TextField(blank=True)
}}}
Create the initial migrations.
{{{
$ python manage.py makemigrations
Migrations for 'core':
core/migrations/0001_initial.py
- Create model Parent
- Create model Child
}}}
Modify the `Child` model in `models.py` so that it no longer subclasses
`Parent`.
{{{#!python
from django.db import models
class Parent(models.Model):
pass
class Child(models.Model):
name = models.TextField(blank=True)
}}}
Create the new migrations. You'll be asked to provide a default value for
the new `id` column. Provide something arbitrary since we'll be modifying
this by hand later to add the data migration:
{{{
$ python manage.py makemigrations core
You are trying to add a non-nullable field 'id' to child without a
default; we can't do that (the database needs something to populate
existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a
null value for this column)
2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can
do e.g. timezone.now
Type 'exit' to exit this prompt
>>> 0
Migrations for 'core':
core/migrations/0002_auto_20200302_1358.py
- Remove field parent_ptr from child
- Add field id to child
}}}
Attempt to check if there are any migrations necessary now. Things will
blow up.
{{{
$ python manage.py makemigrations core
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
....
}}}
This is an issue on 1.11 and 3.0, and presumably everything inbetween.
--
Ticket URL: <https://code.djangoproject.com/ticket/31329>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Stephen Finucane | Owner: nobody
Component: Uncategorized | Version: 3.0
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* status: new => closed
* resolution: => duplicate
Comment:
Marking as a duplicate of #23521.
--
Ticket URL: <https://code.djangoproject.com/ticket/31329#comment:1>