extremely slow django migrations

[Jesse Collier]

Hi all,

First post here. I'm currently trying to transition to django migrations which seems pretty bad ass. We currently do not use south, and keep track of migrations manually (yuk). However, initializing a new database is pretty straight forward; syncdb. It's quick, and efficient. Once we created all our initial migration files, we are noticing extremely slow wall time initializing a database. I've tried profiling and tried various ways of cascading migration files and have fallen short of anything speedy. Has anyone else experienced this? Are there any caveats regarding speed?

Thanks in advance!

-Jesse

[Andrew Pinkham]

Hi Jesse,
Yes, in certain cases, native Django migrations may run very slowly.

https://code.djangoproject.com/ticket/22608

My understanding is that this should not be a problem unless you have a huge number of apps and models (note how the second benchmark from Paul in the ticket has 200 models).

If migrations are running very slowly for you, I would recommend sticking with Django 1.6 and using South first. The upgrade strategy at this point should be:

- upgrade your dependencies to the latest versions (Python, third party apps, etc)
- ensure that the dependencies work with Django 1.7 (Python 2.7, Python 3.2+, etc)
- Attempt upgrade
- test (and fix)
- test again
- do migrations run super slowly? Revert to Django 1.6 and South, wait for Django 1.7.1 (expected fix).

Quick heads up: `syncdb` is deprecated in Django 1.7. If you're using Django 1.7, you should first call `makemigrations`, verify the contents of the file, and then call `migrate` to build/alter the database. However, despite deprecation, `syncdb` should not be having an effect on your migration speeds.

Hope that helps,
Andrew

[John Schmitt]

On Wed, Sep 10, 2014 at 12:08:35PM -0500, Andrew Pinkham wrote:
> Hi Jesse,
...

> Quick heads up: `syncdb` is deprecated in Django 1.7. If you're using Django 1.7, you should first call `makemigrations`, verify the contents of the file, and then call `migrate` to build/alter the database. However, despite deprecation, `syncdb` should not be having an effect on your migration speeds.

Which file? What do I look for? What do I 'verify' and to what should it be compared?

John

[Andrew Pinkham]

On Sep 10, 2014, at 12:25 PM, John Schmitt <marma...@gmail.com> wrote:
> Which file? What do I look for? What do I 'verify' and to what should it be compared?

When you run `makemigrations app_name`, the output will inform you of the creation of at least one migration file, typically in project_name/app_name/migrations. You may look in this(/these) file(s), and ensure that the changes made are the changes you intended on your models.

The documentation allows for understanding of the new files:

https://docs.djangoproject.com/en/1.7/topics/migrations/
https://docs.djangoproject.com/en/1.7/ref/migration-operations/

Note that the verification step is entirely optional. The check is not to ensure that migrations work properly. The check is so that you can check that the changes you made in your models are the ones you intended.

If you've worked with South, the workflow for working with with native migrations is nearly identical.

Hope that helps,
Andrew

[John Schmitt]

On Wed, Sep 10, 2014 at 02:06:09PM -0500, Andrew Pinkham wrote:
> On Sep 10, 2014, at 12:25 PM, John Schmitt <marma...@gmail.com> wrote:
> > Which file? What do I look for? What do I 'verify' and to what should it be compared?
>
> When you run `makemigrations app_name`, the output will inform you of the creation of at least one migration file, typically in project_name/app_name/migrations. You may look in this(/these) file(s), and ensure that the changes made are the changes you intended on your models.
>
> The documentation allows for understanding of the new files:
>
> https://docs.djangoproject.com/en/1.7/topics/migrations/
> https://docs.djangoproject.com/en/1.7/ref/migration-operations/
>
> Note that the verification step is entirely optional. The check is not to ensure that migrations work properly. The check is so that you can check that the changes you made in your models are the ones you intended.

I see. This is exactly what I wanted to know, thank you.

>
> If you've worked with South, the workflow for working with with native migrations is nearly identical.

I've used Python for years but I'm new to Django so I can't relate to South. My references are the official documentation, stackexchange, and this list.