[Django] #24627: Race condition like error in migration
Django
----------------------------+--------------------
Reporter: dwt | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------
Hi there,
I'm having a strange error in some migrations. Let me describe:
At first I had several migrations, one renamed a model process.ProcessDef
to process.ProcessDefinition, then a later one would rename a field on
that model.
{{{
migrations.RenameModel(
old_name='ProcInstance',
new_name='ProcessInstance',
),
migrations.RenameField(
model_name='processinstance',
old_name='status',
new_name='runstatus',
),
}}}
This creates a stack trace that looks like this:
{{{
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/core/management/__init__.py", line 338, in
execute_from_command_line
utility.execute()
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/core/management/commands/migrate.py", line 221, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake,
fake_initial=fake_initial)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/migrations/executor.py", line 147, in apply_migration
state = migration.apply(state, schema_editor)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/migrations/migration.py", line 115, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/migrations/operations/fields.py", line 275, in
database_forwards
to_model._meta.get_field(self.new_name),
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/backends/base/schema.py", line 484, in alter_field
old_db_params, new_db_params, strict)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/backends/sqlite3/schema.py", line 200, in _alter_field
self._remake_table(model, alter_fields=[(old_field, new_field)])
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/backends/sqlite3/schema.py", line 137, in _remake_table
self.create_model(temp_model)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/backends/base/schema.py", line 232, in create_model
definition, extra_params = self.column_sql(model, field)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/backends/base/schema.py", line 131, in column_sql
db_params = field.db_parameters(connection=self.connection)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1989, in db_parameters
return {"type": self.db_type(connection), "check": []}
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1980, in db_type
rel_field = self.related_field
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1886, in related_field
return self.foreign_related_fields[0]
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1620, in
foreign_related_fields
return tuple(rhs_field for lhs_field, rhs_field in
self.related_fields)
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1607, in related_fields
self._related_fields = self.resolve_related_fields()
File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-
packages/django/db/models/fields/related.py", line 1592, in
resolve_related_fields
raise ValueError('Related model %r cannot be resolved' % self.rel.to)
ValueError: Related model 'process.ProcessDef' cannot be resolved
}}}
That is, it seems as if the internal migration logic is missing the fact
that it has just renamed a model, and is trying to look up a relation by
it's old un-renamed name - and then failing.
I have tried to reduce this to one squashed migration, but I'm not sure I
entirely succeeded, as the bug seems to be transient with this migration,
i.e. I get it in about 80-90% of cases, and removing more from it which
*should* be unrelated seems to bring this ratio down quite drastically.
So to sum it up, it feels like a race condition.
Anyway, here it is:
{{{
#!python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
from django.conf import settings
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='FieldDef',
fields=[
('id', models.AutoField(primary_key=True,
verbose_name='ID', serialize=False, auto_created=True)),
('name', models.CharField(max_length=200)),
('descript', models.CharField(max_length=200)),
('fieldhelp', models.CharField(max_length=200)),
('fieldtype', models.PositiveSmallIntegerField()),
('length', models.PositiveSmallIntegerField()),
('editable', models.NullBooleanField()),
('must', models.NullBooleanField()),
('type', models.PositiveSmallIntegerField()),
('parent', models.ForeignKey(to='process.FieldDef')),
],
),
migrations.CreateModel(
name='ProcessDef',
fields=[
('id', models.AutoField(primary_key=True,
verbose_name='ID', serialize=False, auto_created=True)),
('name', models.CharField(max_length=200)),
('descript', models.CharField(max_length=200)),
('status', models.PositiveSmallIntegerField()),
('version', models.PositiveSmallIntegerField()),
('refering', models.ForeignKey(to='process.ProcessDef')),
],
),
migrations.CreateModel(
name='ProcInstance',
fields=[
('id', models.AutoField(primary_key=True,
verbose_name='ID', serialize=False, auto_created=True)),
('starttime', models.DateTimeField()),
('stoptime', models.DateTimeField()),
('status', models.PositiveSmallIntegerField()),
('process', models.ForeignKey(to='process.ProcessDef')),
],
),
migrations.AlterField(
model_name='processdef',
name='refering',
field=models.ForeignKey(to='process.ProcessDef', null=True),
),
migrations.AlterField(
model_name='processdef',
name='refering',
field=models.ForeignKey(to='process.ProcessDef', blank=True,
null=True),
),
migrations.RenameModel(
old_name='ProcessDef',
new_name='ProcessDefinition',
),
migrations.RenameModel(
old_name='ProcInstance',
new_name='ProcessInstance',
),
migrations.RenameField(
model_name='processinstance',
old_name='status',
new_name='runstatus',
),
]
}}}
You can see the project where this is from at
https://github.com/pycess/pycess for reference. There are some old bugs in
the bug database that look kind of similar (e.g. #22319) but not quite,
which is why I am filing this new ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/24627>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: dwt | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.8
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* needs_better_patch: => 0
* resolution: => needsinfo
* needs_tests: => 0
* needs_docs: => 0
Comment:
I couldn't reproduce the error using the provided migration. I ran `python
manage.py migrate` and `python manage.py migrate process zero` several
times in succession without an error. If you could provide a project we
could download to reproduce the error, please reopen. Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/24627#comment:1>