[Django] #24778: Data Migration from Fixture
Django
-----------------------------+--------------------
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+--------------------
Providing data via fixtures has been deprecated. In the past, we used to
execute the loaddata manually. After Django introduce migration, the
recommended way to import data is to create an empty migration and use
`RunPython` migration operations to load the data.
This is a very common use case for data migration via fixture. We create
the function just to call_command `loaddata`.
http://stackoverflow.com/a/25981899/764592
In my opinion, instead of having to create the function, we can actually
simplify this into a migration operation on its own.
As follow:
{{{
# Module: django.db.migrations.operations.base.special
from django.core.management import call_command
class LoadFixture(Operation):
reduces_to_sql = False
reversible = False
def __init__(self, *fixtures):
self.fixtures = fixtures
def state_forwards(self, app_label, state):
pass
def database_forwards(self, app_label, schema_editor, from_state,
to_state):
for fixture in self.fixtures:
call_command('loaddata', fixture, app_label=app_label)
def database_backwards(self, app_label, schema_editor, from_state,
to_state):
pass
def describe(self):
return "Load Fixture Operation"
}}}
The implication of `LoadFixture` operations can be shown in the following
example:
Assuming we have the fixture in `foobar/fixtures/book_data.json`
{{{
# File: foobar/migrations/0002_auto_load_book.py
class Migration(migrations.Migration):
dependencies = [
('foobar', '0001_initial'),
]
operations = [
migrations.LoadFixture('book_data'),
]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24778>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by 9gix):
* cc: 9gix (added)
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
New description:
As follow:
The migration script is now much simpler.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:1>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Description changed by 9gix:
Old description:
> The migration script is now much simpler.
New description:
As follow:
The migration script is now much simpler.
PS: This is my first time creating a ticket and involved in Django
internal. Let me know if I should make a PR for this feature.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:2>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by MarkusH):
* needs_better_patch: 0 => 1
* stage: Unreviewed => Someday/Maybe
* needs_tests: 0 => 1
* needs_docs: 0 => 1
Comment:
I'm a bit torn about this feature. I can understand peoples request to
easily load (existing) fixtures. It's convenient. On the other hand this
will inevitably lead to the very same problem we wanted to prevent by
using `RunPython` in its intended form:
{{{#!python
MyModel = apps.get_model('myapp', 'MyModel')
MyModel.objects.create(...)
}}}
Apart from that, as soon as a model changes your `LoadFixture` operation
will fail: `call_command` will use the model `myapp.models.MyModel`
whereas the database has an older state because the respective changes to
the database would happen in "0003_add_somefield".
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:3>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by charettes):
Markus, what do think about allowing an `apps` kwarg to be passed to the
`loaddata` command and serializers' initializers?
If supplied the command and the serializers would either use the provided
`apps` and default on `django.apps.apps`.
From that point we could provide a `RunPython` subclass that simply calls
`call_command` with `apps=apps`?
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:4>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
That would probably work. I even started going down that rabbit hole of
adding `apps` to the serializers, but revoked the changes because I didn't
see the benefit and it got kind of ugly.
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:5>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
And there are even already apps out there that try to implement fixture
loading during migrations with, of course, the exact same problem I
mentioned before:
* https://github.com/alexhayes/django-migration-fixture
* https://github.com/doctormo/django-fast-fixtures
There even is a library out there that provides that feature for South:
https://github.com/sebleier/django-alpaca
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:6>
Django
Reporter: 9gix | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: master
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by vccabral):
I ran into the app state not matching the model.py state while loading
fixtures and ended up creating a branch of django to handle this use case.
I issued a PR to a feature branch in my fork of django. I am more than
willing to work on this issue given the appropriate guidance. I will
finish up reading the contributor guidelines.
https://github.com/vccabral/django/pull/2
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:7>
Django
Reporter: 9gix | Owner: vccabral
Type: New feature | Status: assigned
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by vccabral):
* owner: nobody => vccabral
* cc: vccabral@… (added)
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:8>
Django
Reporter: 9gix | Owner: vccabral
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
You should probably write to the DevelopersMailingList about this. I'm
uncertain about the design and whether or not it should be included in
Django.
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:9>
Django
Reporter: 9gix | Owner: vccabral
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
As mentioned above in comments 3 to 5 and suggested by charettes, the
"right" approach to this issue is likely adding `apps` as an optional
argument to the serializers and using that in a next step.
A command that dumps a fixture file or a database as a database migrations
feels ugly here. You're essentially creating (arbitrary) python code that
follows absolutely no patterns (how do you handle multi line strings, byte
strings, UUIDs, ...)?
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:10>
Django
Reporter: Eugene | Owner: Victor
Type: New feature | Status: assigned
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
#29801 is a duplicate with some working code.
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:11>
Django
Reporter: Eugene | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by dehnert):
* cc: dehnert (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:12>
Django
Reporter: Eugene | Owner: nobody
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 1 | Needs documentation: 1
Changes (by Jacob Walls):
* cc: Jacob Walls (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/24778#comment:13>