[Django] #25771: Serialization of natural foreign key in migration scripts does not work
Django
----------------------------+----------------------------------------------
Reporter: bowensong | Owner: nobody
Type: Bug | Status: new
Component: | Version: 1.8
Uncategorized |
Severity: Normal | Keywords: database migration serialization
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+----------------------------------------------
The `serializers.serialize(..., use_natural_foreign_keys=True)` is not
working in migration scripts, the `ManyToManyField` always have the value
of the `pk` instead of the `natural_key`
An example to reproduce this bug:
models.py
{{{
# -*- coding: utf-8 -*-
from django.db import models
class Student(models.Model):
student_id = models.CharField(max_length=10, unique=True)
name = models.CharField(max_length=30)
def __unicode__(self):
return self.name
def natural_key(self):
return self.student_id
class Teacher(models.Model):
name = models.CharField(max_length=30)
students = models.ManyToManyField(Student)
def __unicode__(self):
return self.name
}}}
migrations/0002_example.py
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
from django.core import serializers
def example(apps, schema_editor):
Teacher = apps.get_model("mytest", "Teacher")
json_data = serializers.serialize('json', Teacher.objects.all(),
use_natural_foreign_keys=True)
print json_data
class Migration(migrations.Migration):
dependencies = [
('mytest', '0001_initial'),
]
operations = [
migrations.RunPython(example,
reverse_code=migrations.RunPython.noop)
]
}}}
I applied the initial migration, and inserted some data into those tables.
The dumpdata command output is:
{{{
$ python manage.py dumpdata mytest --natural-foreign
[{"fields": {"student_id": "1234567890", "name": "Alice"}, "model":
"mytest.student", "pk": 1}, {"fields": {"student_id": "9876543210",
"name": "Bob"}, "model": "mytest.student", "pk": 2}, {"fields":
{"students": ["1234567890", "9876543210"], "name": "Petter"}, "model":
"mytest.teacher", "pk": 1}]
}}}
When running the migrate command, output is:
{{{
$ python manage.py migrate mytest 0002
Operations to perform:
Target specific migration: 0002_example, from mytest
Running migrations:
Rendering model states... DONE
Applying mytest.0002_example...[{"fields": {"students": [1, 2], "name":
"Petter"}, "model": "mytest.teacher", "pk": 1}]
OK
}}}
Expected output is:
{{{
$ python manage.py migrate mytest 0002
Operations to perform:
Target specific migration: 0002_example, from mytest
Running migrations:
Rendering model states... DONE
Applying mytest.0002_example...[{"fields": {"students": ["1234567890",
"9876543210"], "name": "Petter"}, "model": "mytest.teacher", "pk": 1}]
OK
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25771>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: bowensong | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: database migration | Triage Stage:
serialization | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I think this is expected behavior because the `natural_key()` method isn't
available in migrations. Per
[https://docs.djangoproject.com/en/dev/topics/migrations/#historical-
models the docs], "Because it’s impossible to serialize arbitrary Python
code, these historical models will not have any custom methods that you
have defined."
--
Ticket URL: <https://code.djangoproject.com/ticket/25771#comment:1>
Django
Reporter: bowensong | Owner: nobody
Type: Bug | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Keywords: database migration | Triage Stage:
serialization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => invalid
Comment:
I thought about calling this out in the docs, but I'm not convinced that
doing serialization in migrations is a common use case or something that
we should promote (see #24778 for discussion).
--
Ticket URL: <https://code.djangoproject.com/ticket/25771#comment:2>
Django
Reporter: bowensong | Owner: nobody
Type: Bug | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Keywords: database migration | Triage Stage:
serialization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by bowensong):
OK, my problem was because I'm using the elasticsearch
([https://www.elastic.co/products/elasticsearch]), and elasticsearch uses
JSON format.
In the normal use case, elasticsearch being updated by the `post_save`
signal, but in the event of data migration, the `post_save` function is
not being called. In that case, I have to do the elasticsearch update in
the migration script, and this didn't work because the natural key issue,
I got wrong (unexpected) data in the JSON.
If you are not going to fix it, would you able to provide any suggestions
about how to do this in a correct way?
--
Ticket URL: <https://code.djangoproject.com/ticket/25771#comment:3>
Django
Reporter: bowensong | Owner: nobody
Type: Bug | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Keywords: database migration | Triage Stage:
serialization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Maybe you can define the `natural_key()` method in the migration and then
add it to the model retrieved from the apps registry:
{{{
def natural_key(self):
...
Teacher = apps.get_model("mytest", "Teacher")
Teacher.natural_key = natural_key
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/25771#comment:4>
Django
Reporter: bowensong | Owner: nobody
Type: Bug | Status: closed
Component: Uncategorized | Version: 1.8
Severity: Normal | Resolution: invalid
Keywords: database migration | Triage Stage:
serialization | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by bowensong):
Replying to [comment:4 timgraham]:
> Maybe you can define the `natural_key()` method in the migration and
then add it to the model retrieved from the apps registry:
> {{{
> def natural_key(self):
> ...
>
> Teacher = apps.get_model("mytest", "Teacher")
> Teacher.natural_key = natural_key
> }}}
Thanks, this works perfect.
--
Ticket URL: <https://code.djangoproject.com/ticket/25771#comment:5>