Migrations with multiple databases

[andrea crotti]

We have a couple of databases with a tiny number of tables, but django-migrations has still go through all the migrations anyway.

So even if the SQL itself is nothing it still takes a massive amount of time and memory for the usual known issues with migrations on big projects.

I guess it works this way because it only knows at run time in the router if something should be done, however it's a bit odd.

If we declared the models per database somehow statically, could we not just skip everything else?

[Markus Holtermann]

Hi Andrea,

If you have models that are entirely unrelated in different databases
you could use different values for INSTALLED_APPS to limit the apps
Django sees or by settings the values in the MIGRATION_MODULES dict to
None to tell Django that those apps don't have migrations.

Cheers,

/Markus

>--
>You received this message because you are subscribed to the Google Groups "Django users" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
>To post to this group, send email to django...@googlegroups.com.
>Visit this group at https://groups.google.com/group/django-users.
>To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f5340bbf-2247-4068-bd37-9f3a0faf23d4%40googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.

[andrea crotti]

Ah nice thanks Markus, however 

- different subsets of INSTALLED_APPS sounds a bit hacky, how would I do it anyway?

  checking in the settings which database is being migrated and providing a different list?

  Or any other way?

- MIGRATION_MODULES sounds a bit safer, but I'm not sure it gives the same advantage, since from my understanding django still has to go through all the apps for each db anyway right?

From my understanding the INSTALLED_APPS would be a much bigger performance gain right?

thanks

[marcin....@gmail.com]

On Tuesday, October 25, 2016 at 5:05:46 PM UTC+2, andrea crotti wrote:

We have a couple of databases with a tiny number of tables, but django-migrations has still go through all the migrations anyway.

So even if the SQL itself is nothing it still takes a massive amount of time and memory for the usual known issues with migrations on big projects.

Hi Andrea.

I can suggest you switching to external db migration tool. The Liquibase works for me and my team for years in a quite big projects.

There is a Liquibase adapter for Django - Liquimigrate, which wraps cmd execution using connections defined in settings.DATABASES, and bypasses original "migrate" management commands.

The downside is that you'll have no automatic changeset generation from Django models, you must be familiar with Liquibase changesets syntax (XML, JSON or plain SQL files), and it requires JRE. But the good side is that you have independent, powerfull and rock-solid db migration subsystem with multidb support. It's worth trying.

Marcin