Per package migration history and upgrades

[Mikko Ohtamaa]

Hi,

I am trying to use Alembic in a way that the codebase is split to several Python packages and each package tracks their own tables and migration history independently. Effective each package has its own versions and env.py. Is this possible?  I assume it is possible as Alembic is pretty powerful. I think the major issue here is that each package should have their own migration history table, so they can be tracked and upgraded independently.

Any pointers how I should approach this issue?

Other issues I become aware is that autogenerate consider tables outside the package itself "alien" and tries to drop them. Please see this SO question: http://stackoverflow.com/q/31196631/315168

Just for your information Django migrations handle this kind of situations.

Cheers,
Mikko
https://opensourcehacker.com

[Mike Bayer]

On 7/13/15 12:46 AM, Mikko Ohtamaa wrote:

Hi,

I am trying to use Alembic in a way that the codebase is split to several Python packages and each package tracks their own tables and migration history independently. Effective each package has its own versions and env.py. Is this possible?  I assume it is possible as Alembic is pretty powerful. I think the major issue here is that each package should have their own migration history table, so they can be tracked and upgraded independently.

Any pointers how I should approach this issue?

yes.   use the approach documented at http://alembic.readthedocs.org/en/rel_0_7/branches.html#working-with-multiple-bases.

Other issues I become aware is that autogenerate consider tables outside the package itself "alien" and tries to drop them. Please see this SO question: http://stackoverflow.com/q/31196631/315168

full control over what objects autogenerate considers is present at http://alembic.readthedocs.org/en/rel_0_7/api.html?highlight=include_object#alembic.environment.EnvironmentContext.configure.params.include_object.     if you are trying to run autogenerate in such a way that it considers only individual MetaData objects at a time you probably want to add customization in your env.py file that takes advantage of the "X" argument: http://alembic.readthedocs.org/en/rel_0_7/api.html?highlight=include_object#alembic.environment.EnvironmentContext.get_x_argument.   use this argument to receive which sub-component you want to work on, and consult that within your include_object function to look at just the objects that are relevant to that sub-component.

Just for your information Django migrations handle this kind of situations.

by all means, use Django if it meets your needs.

Cheers,
Mikko
https://opensourcehacker.com

--
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Bayer]

On 7/13/15 11:07 AM, Mike Bayer wrote:

On 7/13/15 12:46 AM, Mikko Ohtamaa wrote:

Hi,

I am trying to use Alembic in a way that the codebase is split to several Python packages and each package tracks their own tables and migration history independently. Effective each package has its own versions and env.py. Is this possible?  I assume it is possible as Alembic is pretty powerful. I think the major issue here is that each package should have their own migration history table, so they can be tracked and upgraded independently.

Any pointers how I should approach this issue?

yes.   use the approach documented at http://alembic.readthedocs.org/en/rel_0_7/branches.html#working-with-multiple-bases.

let me revise that as since you said each project has its own env.py and you'd like them to have separate alembic version tables, you probably want to use the old approach here.   Do this by setting the version_table variable in your EnvironmentContext.configure(): http://alembic.readthedocs.org/en/rel_0_7/api.html#alembic.environment.EnvironmentContext.configure.params.version_table

in order to have multiple env.py environments referenced in a single config, specify a [section] in your alembic.ini file local to each one and use the "-n" argument to the "alembic" command:

   -n NAME, --name NAME  Name of section in .ini file to use for Alembic config

within each [section], specify a different script_location:


[env_1]

script_location = /path/to/env1

[env_2]

script_location = /path/to/env2

then to run upgrades on env_2 you'd say:

    alembic -n env_2 upgrade head


since you have separate env.py scripts, you can put an include_objects function in each one to manage the scope of autogenerate.

[Mikko Ohtamaa]

Hi Mike,

let me revise that as since you said each project has its own env.py and you'd like them to have separate alembic version tables, you probably want to use the old approach here.   Do this by setting the version_table variable in your EnvironmentContext.configure(): http://alembic.readthedocs.org/en/rel_0_7/api.html#alembic.environment.EnvironmentContext.configure.params.version_table

> since you have separate env.py scripts, you can put an include_objects function in each one to manage the scope of autogenerate.

Perfect! include_object was exactly what I was looking for.

Thank you for a great project.

-Mikko

Other issues I become aware is that autogenerate consider tables outside the package itself "alien" and tries to drop them. Please see this SO question: http://stackoverflow.com/q/31196631/315168

full control over what objects autogenerate considers is present at http://alembic.readthedocs.org/en/rel_0_7/api.html?highlight=include_object#alembic.environment.EnvironmentContext.configure.params.include_object.     if you are trying to run autogenerate in such a way that it considers only individual MetaData objects at a time you probably want to add customization in your env.py file that takes advantage of the "X" argument: http://alembic.readthedocs.org/en/rel_0_7/api.html?highlight=include_object#alembic.environment.EnvironmentContext.get_x_argument.   use this argument to receive which sub-component you want to work on, and consult that within your include_object function to look at just the objects that are relevant to that sub-component.

Just for your information Django migrations handle this kind of situations.

by all means, use Django if it meets your needs.

Cheers,
Mikko
https://opensourcehacker.com

--
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sqlalchemy-alembic/rQM8auJUfVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sqlalchemy-alem...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mikko Ohtamaa
http://opensourcehacker.com
http://twitter.com/moo9000