Bypassing 1.7+ migrations

[Marcin Nowak]

Hello!

I'm using Django as a part of bigger stack where I'm using external tool to manage database migrations.

After migration to Django 1.7 I'm encouraged at every runserver command to execute some migrations, which may blow my databases.

I want to bypass Django migrations to prevent any schema modification. I can bypass migrate*  commands by overriding them in my custom app, but this is a little tricky and does not cover all cases. Using managed=False is not a solution, because none of reusable apps have models defined as unmanaged (same for django.contrib, especially auth and sessions).

That's why I need to bypass migrations globally.

Have you ever considered possibility to allow disabling it? Maybe via some setting to use kind a DummyMigrationExecutor instead of MigrationExecutor? 

Cheers,

Marcin 

[Andrew Godwin]

Hi Marcin,

If you're using an external tool to manage schemas of models, just set managed=False on the models and Django will stop trying to change their schemas (including stopping making migrations for them).

Andrew

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a27293a1-4212-4a2a-ae44-4720fc674162%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marcin Nowak]

Hi Andrew,

Thanks for a reply. But as I've mentioned above I can't set managed=False on models delivered from external apps like django.contrib.auth, django.contrib.sessions, and many more. Forking them is not an option.

I think that bypassing migrations should be possible via some setting. That's why I wrote to developers forum directly.

Please think about that.

Marcin

[Andrew Godwin]

Hi Marcin,

You can't bypass migrations as much as you can't bypass syncdb; that is, the commands are separate and you can choose not to run them (or just delete the migration files). As far as I can tell your only issue is the runserver warning that you have unmigrated changes, which there's currently no plan to be able to disable.

Also bear in mind that, by doing your own schema management for the contrib apps, you're kind of implicitly forking the schema; Django 1.8 will ship with some migrations for these apps, and you're going to have to make sure you manage those yourself and keep the database in sync with the code (though of course the release notes will mention this kind of thing).

If you want to stub out migration operations, you can just make a custom database backend that inherits from the one you use and define a new SchemaEditor subclass with an execute() function that does nothing; that should accomplish what you want. Alternatively, you could wrap all migration operations into the State side of SeparateDatabaseAndState, but that's a bit cumbersome.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d8226a81-9dfe-4019-94ce-98beb6e1bf04%40googlegroups.com.

[Markus Holtermann]

Hey Marcin, Andrew,

setting manager=False wouldn't work either, as the migrations for those models still need to exist. I probably have references to them which makes them a requirement for FKs in migrations. Unmanaged models behave exactly the same as managed models in migrations, with one exception: the database operations aren't run.

/Markus

On Tuesday, December 16, 2014 1:34:17 PM UTC+1, Andrew Godwin wrote:

[Marcin Nowak]

On Tuesday, December 16, 2014 1:49:53 PM UTC+1, Andrew Godwin wrote:

Also bear in mind that, by doing your own schema management for the contrib apps, you're kind of implicitly forking the schema; Django 1.8 will ship with some migrations for these apps, and you're going to have to make sure you manage those yourself and keep the database in sync with the code (though of course the release notes will mention this kind of thing).

Yes, I'm aware of that, and I want to control every schema change (this is specific requirement for this project) 

 

If you want to stub out migration operations, you can just make a custom database backend that inherits from the one you use and define a new SchemaEditor subclass with an execute() function that does nothing; that should accomplish what you want.

Thanks! I will try this approach. 

 

Alternatively, you could wrap all migration operations into the State side of SeparateDatabaseAndState, but that's a bit cumbersome.

For this I need to do some research. 

Thanks for tips!

 As far as I can tell your only issue is the runserver warning that you have unmigrated changes, which there's currently no plan to be able to disable.

I would like to bypass migrations to prevent accidental execution of `migrate` command, too.   

Generally speaking - I don't need builtin migrations (because executing them in my environment may be dangerous), but I need newer versions of Django.

I will try your tips.

 

Kind Regards,

Marcin

[Shai Berger]

FWIW, the approach I would go for is to <handwaving>try to monkeypatch
managed=False into the base model Meta</handwaving>.

Shai.

> >>> <https://groups.google.com/d/msgid/django-developers/a27293a1-4212-4a2a
> >>> -ae44-4720fc674162%40googlegroups.com?utm_medium=email&utm_source=foote
> >>> r> .

> >>> For more options, visit https://groups.google.com/d/optout.
> >>>
> >> --
> >
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-develop...@googlegroups.com.
> > To post to this group, send email to django-d...@googlegroups.com.
> > Visit this group at http://groups.google.com/group/django-developers.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/django-developers/d8226a81-9dfe-4019-94
> > ce-98beb6e1bf04%40googlegroups.com

> > <https://groups.google.com/d/msgid/django-developers/d8226a81-9dfe-4019-
> > 94ce-98beb6e1bf04%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .

[Markus Holtermann]

Hey Shai,

manage=False won't gain you anything, since those models will still show
up in migrations, because they need to be able to be referenced by
managed models. The migration database operations are noops though.

/Markus

--

[Shai Berger]

Hi Markus,

On Thursday 18 December 2014 08:54:30 Markus Holtermann wrote:
>
> manage=False won't gain you anything, since those models will still show
> up in migrations, because they need to be able to be referenced by
> managed models. The migration database operations are noops though.
>

As far as I understood, Marcin wants to treat *all* models as unmanaged; there
are no managed models to reference the unmanaged ones.

Shai.

[Marcin Nowak]

On Friday, December 19, 2014 12:19:11 AM UTC+1, Shai Berger wrote:

As far as I understood, Marcin wants to treat *all* models as unmanaged; there
are no managed models to reference the unmanaged ones.

I would like to disable/remove/kick-off built-in migration mechanism. :)

If may I suggest something - the migration subsystem should be delivered in the same way as other "contrib" apps. 

It should be an extension similar to South, but delivered as a part and of framework.

 

/Marcin

[Carl Meyer]

The "syncdb" functionality is on its way to deprecation, replaced by
migrations, so at some point within a few releases migrations will be
the only way Django can translate your models into database schema. This
is not an optional contrib add-on, it's a core part of the ORM.

That said, if you don't want to use migrations, you don't have to - it
just means that you are 100% responsible for creating and modifying your
necessary database tables. That's what setting `managed = False` on the
`Meta` of all your models will accomplish.

Carl

[Collin Anderson]

Hi All,

Totally a shot in the dark here, but would allow_migrate work in this case?

https://docs.djangoproject.com/en/dev/topics/db/multi-db/#allow_migrate

Collin

[Marcin Nowak]

 This is not an optional contrib add-on, it's a core part of the ORM.

There are systems where Django is not a core, but it is used as lightweight app server.

Database schema migrations are created and executed in the other way.. 

That's why is so important to not change any database schema.

   

That said, if you don't want to use migrations, you don't have to

Yes, of course. But somebody can still hit <enter> after "migration" command and execute something very bad.

To prevent mistakes like this I would like to disable migrations for specified databases or for whole Django-based project.

 

 That's what setting `managed = False` on the `Meta` of all your models will accomplish.

Yes, I'll try this. 

It should help me in my case, but this is not same as bypassing migrations subsystem. 

Built-in migrations should be optional.

 

Kind Regards,

Marcin

[Carl Meyer]

Hi Marcin,

On 12/22/2014 10:28 AM, Marcin Nowak wrote:
>> This is not an optional contrib add-on, it's a core part of the ORM.
>
> There are systems where Django is not a core, but it is used as lightweight
> app server.
> Database schema migrations are created and executed in the other way..
> That's why is so important to not change any database schema.

Yes - that's precisely the scenario for which managed=False is intended.

>> That said, if you don't want to use migrations, you don't have to
>
>
> Yes, of course. But somebody can still hit <enter> after "migration"
> command and execute something very bad.
> To prevent mistakes like this I would like to disable migrations for
> specified databases or for whole Django-based project.

If you set managed=False on the models, no such mistake can occur.

>> That's what setting `managed = False` on the `Meta` of all your models
>> will accomplish.
>>
>>
> Yes, I'll try this.
> It should help me in my case, but this is not same as bypassing migrations
> subsystem.
> Built-in migrations should be optional.

I'm not sure I understand what you mean by "bypassing migrations
subsystem", then. Setting managed=False is the same as bypassing migrations.

Either some other system is responsible for creating and maintaining
database schema, or Django is responsible for it.

If some other system is responsible (as in your case), then you should
use managed=False and Django will not touch the db schema, ever.

If Django is responsible (as for most users), then migrations are now
the _only_ (non-deprecated) method Django uses to create or modify
database schema.

The one feature I can see that might be missing is a way to set the
equivalent of managed=False globally, so that it also applies to
third-party apps (without having to monkeypatch them).

Carl