running migrations programmatically

[tiadobatima]

Hello there,

I'm trying to create one framework to execute alembic commands programmatically.

Basically I'm trying to do away with alembic.ini, env.py and all other files but the actual migration scripts.

The reason is that I have several applications using the same DB framework with its own DB configuration and having the DB configuration in two different places is not a good idea.

I'd like to pass the 'script_location'  and 'sqlalchemy.url' to the framework from my own configuration files to create the alembic config, and then to be able to run alembic commands, like stamp, upgrade, downgrade using just the standard template.

I was able to create my a minimum config object ( with 'script_location'  and 'sqlalchemy.url' only) and created this function to execute alembic commands:

    def migrate_tables(self, command, alembic_config=None, **kwargs):

        alembic_config = alembic_config or self.alembic_config

        if not alembic_config:

            raise ValueError('Could not resolve the alembic configuration')

        cmd = getattr(alembic.command, command)

        return cmd(config=alembic_config, **kwargs)

But I can't really figure out how to get rid of env.py, or at least have only one env.py in the framework, while using the different script directories for different applications.

Any ideas are greatly appreciated :)

Cheers!

g.

[Michael Bayer]

On Dec 10, 2013, at 12:46 PM, tiadobatima <gbar...@gmail.com> wrote:

> Hello there,
>
> I'm trying to create one framework to execute alembic commands programmatically.
> Basically I'm trying to do away with alembic.ini, env.py and all other files but the actual migration scripts.
>
> The reason is that I have several applications using the same DB framework with its own DB configuration and having the DB configuration in two different places is not a good idea.
> I'd like to pass the 'script_location' and 'sqlalchemy.url' to the framework from my own configuration files to create the alembic config, and then to be able to run alembic commands, like stamp, upgrade, downgrade using just the standard template.

you can run the alembic op.XYZ() methods without having alembic.ini or env.py present. but running the alembic commands themselves in alembic.command, those are hardwired to the alembic.script.ScriptDirectory concept which itself is hardwired to env.py. So if you want to run the script files without env.py specifically, you either have to subclass/monkeypatch ScriptDirectory somehow or you need to rewrite what the commands in alembic.command actually do.

I don’t actually understand the issue, “several applications using the same DB framework”, and then you have some special way of getting at the database url, OK, that’s exactly why env.py is provided in the first place - you’re supposed to customize it to do whatever you need in order to get at the configuration. for example see the “pylons” example template that pulls the database config from Pylons’ config objects. env.py is there for you to put whatever you want inside of it, it can import modules or do anything you need it to.

[tiadobatima]

Thanks for the quick answer Michael :)

Unless I'm still not understanding how it works, the way it is now we need one env.py per application - or set of migration scripts (ScriptDirectory). 

We don't really need/want a custom env.py for each app, in fact we'd prefer if DB migrations are performed exactly the same across all apps.

So, I guess I was looking for a way to decouple env.py from alembic.script.ScriptDirectory. 

Maybe a way to set the path to env.py in ScriptDirectory, or in the Config object, so run_env() could run an arbitrary env.py. No way we can do this easily right?

Thank you very much again :)

g.

[Michael Bayer]

On Dec 10, 2013, at 2:54 PM, tiadobatima <gbar...@gmail.com> wrote:

Thanks for the quick answer Michael :)

Unless I'm still not understanding how it works, the way it is now we need one env.py per application - or set of migration scripts (ScriptDirectory). 

We don't really need/want a custom env.py for each app, in fact we'd prefer if DB migrations are performed exactly the same across all apps.

So, I guess I was looking for a way to decouple env.py from alembic.script.ScriptDirectory. 

Maybe a way to set the path to env.py in ScriptDirectory, or in the Config object, so run_env() could run an arbitrary env.py. No way we can do this easily right?

I’m really not understanding the setup here.  Whereever you have a set of migration files, that’s where the env.py is.  It can be just one line long: “from mylibrary.shared_env import *”.    I have an app here with several sub-applications, each with their own sets of migration files, which uses this approach - the things that are specific to the app go in the local env.py, it then pulls everything else from a common env.py as far as how to run migrations.    I use the “name” argument to alembic so that each alembic config can all be rolled up into one .ini file in different sections too.      It’s pretty open ended.

Thank you very much again :)

g.

On Tuesday, 10 December 2013 11:16:54 UTC-8, Michael Bayer wrote:

On Dec 10, 2013, at 12:46 PM, tiadobatima <gbar...@gmail.com> wrote: 

> Hello there, 
> 
> I'm trying to create one framework to execute alembic commands programmatically. 
> Basically I'm trying to do away with alembic.ini, env.py and all other files but the actual migration scripts. 
> 
> The reason is that I have several applications using the same DB framework with its own DB configuration and having the DB configuration in two different places is not a good idea. 
> I'd like to pass the 'script_location'  and 'sqlalchemy.url' to the framework from my own configuration files to create the alembic config, and then to be able to run alembic commands, like stamp, upgrade, downgrade using just the standard template. 

you can run the alembic op.XYZ() methods without having alembic.ini or env.py present.  but running the alembic commands themselves in alembic.command, those are hardwired to the alembic.script.ScriptDirectory concept which itself is hardwired to env.py.  So if you want to run the script files without env.py specifically, you either have to subclass/monkeypatch ScriptDirectory somehow or you need to rewrite what the commands in alembic.command actually do. 

I don’t actually understand the issue, “several applications using the same DB framework”, and then you have some special way of getting at the database url, OK, that’s exactly why env.py is provided in the first place - you’re supposed to customize it to do whatever you need in order to get at the configuration.  for example see the “pylons” example template that pulls the database config from Pylons’ config objects.   env.py is there for you to put whatever you want inside of it, it can import modules or do anything you need it to. 

-- 
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Felix Schwarz]

Hey,

besides what Mike already said you might want to have a look at MediaDrop (+
SEO plugin) where I built similar functionality. For example "paster setup-app
deployment.ini" (besides doing other things) runs all migrations (including
migrations from plugins.

I usually keep an alembic.ini anyway because my integration does not expose
all functionality of the alembic command line tool so it is convenient to use
"alembic" when building the migration script.

fs