Best Practices for Migrations in Reusable Apps?

[Jonathan Barratt]

Hi All,

I'm interested to know what the community's opinions are on how best to package migrations with reusable apps (I'm thinking of the built-in migrations, but it seems the same principles would apply for pre-1.8 south migrations).

My current approach is to squash all schema migrations each time a new one is generated so that new installs are smooth; we've seen issues arise from just keeping the individual migrations that accumulate the long way (in terms of both speed and bugs).

I figure existing users who upgrade can be expected to run their own make migrations if necessary, but perhaps I'm placing the burden of work on the wrong party...

If initial data is necessary, then it gets its own separate data migration - this seems to keep the code (and git history) cleaner IMO.

I'd greatly appreciate and criticism, suggestions or alternate approaches that occurs to you.

Thanks in advance,
Jonathan

[Avraham Serour]

usually people will upgrade using pip install -U

if your lib needs more than that it should be the least complicated possible, I believe making people make their own migrations would be a burden

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/146C30FE-903F-4823-916F-F50B03DFDCB5%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Carl Meyer]

Hi Jonathan,

On 06/10/2015 12:24 AM, Jonathan Barratt wrote:
> I'm interested to know what the community's opinions are on how best
> to package migrations with reusable apps (I'm thinking of the
> built-in migrations, but it seems the same principles would apply for
> pre-1.8 south migrations).
>
> My current approach is to squash all schema migrations each time a
> new one is generated so that new installs are smooth;

You can do this, but you should keep the old migrations around as well,
for the sake of your existing upgrading users. You should only remove
replaced migrations once you are confident that all your users (or
whatever percentage of them you care about) have upgraded.

The migration-squashing system is designed to make it easy to squash
migrations for new users but also keep the old ones around for upgrading
users; it automatically takes care of making sure the right migrations
are used in each case.

> we've seen
> issues arise from just keeping the individual migrations that
> accumulate the long way (in terms of both speed and bugs).

Speed can be an issue with lots of migrations; squashing migrations (but
keeping the old ones) addresses this.

Not much that can be said about bugs without specific examples. The
migration system has certainly had its share of bugs! Almost all the
ones we know about have been fixed. Your old set of migrations should
result in the exact same schema as the new squashed one. If it doesn't,
that's certainly a problem that needs to be fixed, in Django and/or in
one or more of the migration files.

> I figure existing users who upgrade can be expected to run their own
> make migrations if necessary, but perhaps I'm placing the burden of
> work on the wrong party...

This is placing the burden on the wrong party (and on many parties,
instead of one party), and it actually puts your upgrading users in
quite a pickle. They can't add migrations directly to your codebase
without forking it, which means their only recourse to add the necessary
migration is to set MIGRATION_MODULES to ignore your migrations entirely
and replace them with their own.

MIGRATION_MODULES is intended for rare cases, not routine use. Your app
should contain the necessary migrations for all your users, new and
upgrading. Failing to provide migrations that will work for upgrading
users means you miss out on all the benefits of migrations; you're
essentially returning to the old syncdb world, where no provision is
made for upgrades that include schema changes.

> If initial data is necessary, then it gets its own separate data
> migration - this seems to keep the code (and git history) cleaner
> IMO.
>
> I'd greatly appreciate and criticism, suggestions or alternate
> approaches that occurs to you.
>
> Thanks in advance, Jonathan

Hope that helps,

Carl

[reduxionist]

I was afraid of that, glad I asked. Thanks for your input!

[reduxionist]

Wow, terrific and comprehensive answer (as usual) Carl, thanks! I'll adjust my approach accordingly, thanks very much!