[Django] #24630: UUIDField migration
Django
----------------------------+----------------------------------
Reporter: lucacorti | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Severity: Normal | Keywords: uuidfiled migrations
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+----------------------------------
As per #23932, I added an UUIDField to a model following the documentation
at https://docs.djangoproject.com/en/1.8/howto/writing-migrations/.
When running the migration I get this error:
{{{
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "venv/lib/python2.7/site-
packages/django/core/management/__init__.py", line 338, in
execute_from_command_line
utility.execute()
File "venv/lib/python2.7/site-
packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "venv/lib/python2.7/site-packages/django/core/management/base.py",
line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "venv/lib/python2.7/site-packages/django/core/management/base.py",
line 441, in execute
output = self.handle(*args, **options)
File "venv/lib/python2.7/site-
packages/django/core/management/commands/migrate.py", line 221, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "venv/lib/python2.7/site-
packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake,
fake_initial=fake_initial)
File "venv/lib/python2.7/site-
packages/django/db/migrations/executor.py", line 147, in apply_migration
state = migration.apply(state, schema_editor)
File "venv/lib/python2.7/site-
packages/django/db/migrations/migration.py", line 115, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "venv/lib/python2.7/site-
packages/django/db/migrations/operations/fields.py", line 201, in
database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "venv/lib/python2.7/site-
packages/django/db/backends/base/schema.py", line 484, in alter_field
old_db_params, new_db_params, strict)
File "venv/lib/python2.7/site-
packages/django/db/backends/base/schema.py", line 637, in _alter_field
params,
File "venv/lib/python2.7/site-
packages/django/db/backends/base/schema.py", line 107, in execute
cursor.execute(sql, params)
File "venv/lib/python2.7/site-packages/django/db/backends/utils.py",
line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "venv/lib/python2.7/site-packages/django/db/backends/utils.py",
line 64, in execute
return self.cursor.execute(sql, params)
File "venv/lib/python2.7/site-packages/django/db/utils.py", line 97, in
__exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "venv/lib/python2.7/site-packages/django/db/backends/utils.py",
line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.OperationalError: cannot ALTER TABLE "mytable" because it
has pending trigger events
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24630>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by lucacorti):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
New description:
As per #23932, I added an UUIDField to a model following the documentation
at https://docs.djangoproject.com/en/1.8/howto/writing-migrations/. The
database is PostgreSQL.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by KevinEtienne):
I've run into what I believe the same issue with Django 1.7.x. I have a
model where I wanted to add and remove a new field.
The migration had 3 operations:
* add a new field
* migrate the data
* and remove the old field.
{{{
...
operations = [
migrations.AddField(
model_name='model',
name='new_field',
field=models.BooleanField(default=False),
preserve_default=True,
),
migrations.RunPython(
populate_new_field,
populate_old_field,
),
migrations.RemoveField(
model_name='model',
name='old_field',
),
]
}}}
The stacktrace looks exactly the same as quote in the original
description. Not sure if this is a behavior expected from Django or not.
[https://docs.djangoproject.com/en/1.8/ref/migration-operations/#runpython
The documentation on 1.7] and 1.8 states that `RunPython` should run into
its own transaction.
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by darjeeling):
your database is postgresql?
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
Type: Bug | Status: assigned
Component: Migrations | Version: 1.8
Severity: Normal | Resolution:
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by priidukull):
* status: new => assigned
* owner: nobody => priidukull
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Type: Bug | Status: assigned
Severity: Normal | Resolution:
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by priidukull):
I tried to reproduce the bug with Python 2.7 and Django 1.7/1.8. Did not
manage to reproduce the issue. I used the following migrations.
0001_initial.py:
{{{#!python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
]
operations = [
migrations.CreateModel(
name='Choice',
fields=[
('id', models.AutoField(verbose_name='ID',
serialize=False, auto_created=True, primary_key=True)),
('choice_text', models.CharField(max_length=200)),
('votes', models.IntegerField(default=0)),
],
),
migrations.CreateModel(
name='Question',
fields=[
('id', models.AutoField(verbose_name='ID',
serialize=False, auto_created=True, primary_key=True)),
('question_text', models.CharField(max_length=200)),
('pub_date', models.DateTimeField(verbose_name=b'date
published')),
],
),
migrations.AddField(
model_name='choice',
name='question',
field=models.ForeignKey(to='polls.Question'),
),
]
}}}
0002_second.py:
{{{#!python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
import random
from django.db import migrations
def populate_old_field(apps, schema_editor):
Question = apps.get_model('polls', 'Question')
for i in range(1, 100):
q = Question(i, random.getrandbits(128), '2015-09-09')
q.save()
class Migration(migrations.Migration):
dependencies = [
('polls', '0001_initial'),
]
operations = [
migrations.RunPython(
populate_old_field
),
]
}}}
0003_third.py:
{{{#!python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def populate_new_field(apps, schema_editor):
Question = apps.get_model('polls', 'Question')
for q in Question.objects.all():
q.question_text_new = q.question_text
q.save()
class Migration(migrations.Migration):
dependencies = [
('polls', '0002_second'),
]
operations = [
migrations.AddField(
model_name='question',
name='question_text_new',
field=models.CharField(max_length=300, null=True),
),
migrations.RunPython(
populate_new_field,
),
migrations.RemoveField(
model_name='question',
name='question_text',
),
]
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Migrations | Version: 1.8
Severity: Normal | Resolution: needsinfo
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by priidukull):
* status: assigned => closed
* resolution: => needsinfo
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Type: Bug | Status: closed
Severity: Normal | Resolution: needsinfo
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by priidukull):
* cc: priidukull (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Type: Bug | Status: closed
Severity: Normal | Resolution: needsinfo
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by ollanta):
Hi, I've stumbled across this issue as well.
I'm not getting the error with your example either priidukull, so I
reduced the code I had to a small example that still exhibits the bug (in
both django 1.7 and 1.8, but only in psql, not sqlite)
0001_initial.py :
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
from django.conf import settings
class Migration(migrations.Migration):
dependencies = []
operations = [
migrations.CreateModel(
name='Parent',
fields=[
('id', models.AutoField(verbose_name='ID',
serialize=False, auto_created=True, primary_key=True)),
],
),
migrations.CreateModel(
name='Child',
fields=[
('id', models.AutoField(verbose_name='ID',
serialize=False, auto_created=True, primary_key=True)),
('boolfield', models.BooleanField(default=True)),
('parent', models.ForeignKey(to='example.Parent')),
],
),
]
}}}
0002_add_data.py:
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def add_data(apps, schema_editor):
Parent = apps.get_model("example", "Parent")
p = Parent(1)
p.save()
Child = apps.get_model("example", "Child")
c = Child(1, parent=p, boolfield=True)
c.save()
class Migration(migrations.Migration):
dependencies = [
('example', '0001_initial'),
]
operations = [
migrations.RunPython(
add_data
)
]
}}}
0003_change_field_type.py:
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def convert_field(apps, schema_editor):
Child = apps.get_model("example", "Child")
for c in Child.objects.all():
c.textchoicefield = 'true' if c.boolfield else 'false'
c.save()
class Migration(migrations.Migration):
dependencies = [
('example', '0002_add_data'),
]
operations = [
migrations.AddField(
model_name='child',
name='textchoicefield',
field=models.TextField(default=b'none', choices=[(b'true',
b'True'), (b'false', b'False'), (b'none', b'None')]),
preserve_default=True,
),
migrations.RunPython(convert_field),
migrations.RemoveField(
model_name='child',
name='boolfield',
),
]
}}}
If I move RemoveField out of the last migration and into its own, it all
works fine.
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Migrations | Version: 1.8
Keywords: uuidfiled | Triage Stage:
migrations | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: closed => new
* resolution: needsinfo =>
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Documentation | Version: 1.8
Severity: Release blocker | Resolution:
Keywords: uuidfiled | Triage Stage: Accepted
migrations |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Unreviewed => Accepted
* component: Migrations => Documentation
* severity: Normal => Release blocker
Comment:
It seems to me that the documentation that says, "By default, RunPython
will run its contents inside a transaction" is misleading (added in
5a917cfef319df33ca30a3b27bd0b0533b8e63bb).
Each *migration* is run in its own transaction (if the database backend
has `can_rollback_ddl = True`) because we use
[https://github.com/django/django/blob/1418f7530455abf3e6f743164ae15ee6b2566a3c/django/db/migrations/executor.py#L122-L128
a SchemaEditor context manager] when applying each migration, which
[https://github.com/django/django/commit/e74d2183c28467aefc0b87e3fa6d405dbfdea82c
creates a transaction].
I'm not sure we can make any changes code-wise to support mixing schema
changes and RunPython operations in the same migration, but we should
correct the documentation for current version of Django anyway.
Docs to fix:
* The second example migration needs to be split into at least two
migrations. https://docs.djangoproject.com/en/1.8/howto/writing-migrations
/#migrations-that-add-unique-fields
* "By default, RunPython will run its contents..."
https://docs.djangoproject.com/en/1.8/ref/migration-operations/#runpython
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Documentation | Version: 1.8
Severity: Release blocker | Resolution:
Keywords: uuidfiled | Triage Stage: Accepted
migrations |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/4650 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Documentation | Version: 1.8
Severity: Normal | Resolution:
Keywords: uuidfiled | Triage Stage: Accepted
migrations |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* severity: Release blocker => Normal
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Documentation | Version: 1.8
Severity: Normal | Resolution:
migrations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by MarkusH):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Component: Documentation | Version: 1.8
Severity: Normal | Resolution: fixed
Keywords: uuidfiled | Triage Stage: Ready for
migrations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"307acc745a4e655c35db96f96ceb4b87597dee49" 307acc7]:
{{{
#!CommitTicketReference repository=""
revision="307acc745a4e655c35db96f96ceb4b87597dee49"
Fixed #24630 -- Clarified docs about RunPython transactions.
Thanks Markus Holtermann for review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: lucacorti | Owner:
| priidukull
Type: Bug | Status: closed
Component: Documentation | Version: 1.8
Severity: Normal | Resolution: fixed
Keywords: uuidfiled | Triage Stage: Ready for
migrations | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"cc4ee06279a408d033e30bade774f3c6715bfe3e" cc4ee062]:
{{{
#!CommitTicketReference repository=""
revision="cc4ee06279a408d033e30bade774f3c6715bfe3e"
[1.8.x] Fixed #24630 -- Clarified docs about RunPython transactions.
Thanks Markus Holtermann for review.
Backport of 307acc745a4e655c35db96f96ceb4b87597dee49 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24630#comment:15>