Issue 460 in flyway: Allow multiple scripts per version

[fly...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Enhancement Priority-Medium Usability

New issue 460 by torr0...@gmail.com: Allow multiple scripts per version
http://code.google.com/p/flyway/issues/detail?id=460

I have a large schema with a large seed data set. It is unmanageable (I can
barely open an editor) to have this in one file. We have traditionally
broken this up into one sql file per table. However, Flyway doesn't support
this.

I know I can work around this by versioning each script as a pre-1.0
script, but that's not accurate. Ideally, Flyway would support multiple
files at the same version, but with different names, so that we can migrate
to a 1.0 and have the scripts match.



--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[fly...@googlecode.com]

Updates:
Status: WontFix

Comment #1 on issue 460 by axel.fontai...@gmail.com: Allow multiple

scripts per version
http://code.google.com/p/flyway/issues/detail?id=460

Hi Todd,

Thanks for your suggestion. Your workaround is the right way to go. Divide
them up in 0.9.x (or whatever other number with point releases) and you
should be good to go. Multiple files per version would basically require
emulating this internally without providing any real value. (And it would
create new problems, like finding a reproducible order to run them in)

Cheers
Axel

[fly...@googlecode.com]

Comment #2 on issue 460 by torr0...@gmail.com: Allow multiple scripts per
version
http://code.google.com/p/flyway/issues/detail?id=460

I think that this workaround is not a good user experience. I implore you
to rethink this "Won't Fix". The workaround is confusing to new users and
is sure to get messed up in a large enough team. Also, the workaround is
just plain misleading because it forces you to name a script for a version
to which it doesn't belong.

[fly...@googlecode.com]

Comment #3 on issue 460 by virid...@gmail.com: Allow multiple scripts per
version
http://code.google.com/p/flyway/issues/detail?id=460

I agree. For modular database support, where the scripts are co-located
with their module, it becomes difficult to manage versions. It is
unreasonable for all the scripts to be colocated just so we avoid version
clashes.

[fly...@googlecode.com]

Comment #4 on issue 460 by m.nuess...@googlemail.com: Allow multiple

scripts per version
http://code.google.com/p/flyway/issues/detail?id=460

If Flyway does not support to split up the changes for a version into
multiple files, it becomes unmanageable in my case too. The suggested
workaround is not very appealing. Flyway looked promising otherwise, but
this is a no-go.

[fly...@googlecode.com]

Comment #5 on issue 460 by torr0...@gmail.com: Allow multiple scripts per
version
http://code.google.com/p/flyway/issues/detail?id=460

Unfortunately, the maintainer is adamant about this not being a good
feature. I even inquired about paying for this feature, but the query was
not entertained. I think the only option is to fork it, but, for myself,
I'm not inclined to do so and have been investigating other options.

[fly...@googlecode.com]

Comment #6 on issue 460 by axel.fontai...@gmail.com: Allow multiple

scripts per version
http://code.google.com/p/flyway/issues/detail?id=460

Hi guys,

sorry to hear you're disappointed by this. I've voiced my concern in my
initial reply. So far, it has not been addressed.

Todd is correct. No amount of money will make me change my mind as long as
no suitable solution exists. It is real hard work to keep a project like
Flyway from drifting into unmanageable complexity. And sometimes this means
having to say no.

Cheers
Axel

[fly...@googlecode.com]

Comment #7 on issue 460 by virid...@gmail.com: Allow multiple scripts per
version
http://code.google.com/p/flyway/issues/detail?id=460

Look at the powerful concept called "Package by Feature":
http://www.javapractices.com/topic/TopicAction.do?Id=205

I have promoted this approach to my team and they love it. Reducing
coupling, enhancing cohesion, limiting class visibility, what more can you
say. This is a Good Thing. The problem seems to be that Flyway fights this
model, inviting you to bring your scripts together to make sure you haven't
messed up your versions.

Flyway could support multiple scripts with the same version. Ordering could
be resolved by convention. You could say the order of execution for scripts
having the same version is undefined. This feature would need to be
explicitly enabled for "advanced" usages only.

You can also use multiple instances of Flyway behind a custom-managed
module API, making each instance manage a separate package, with a separate
meta table. In fact, I have a working example of this approach.

Even better (skip the module APIs and multiple instances) is to stick with
a single instance and require that versions be decorated with the module
name. Flyway should work with this.

So instead of:
com/mycompany/feature1/1.0.0__Initialization.sql
com/mycompany/feature1/1.0.1__Updates.sql
com/mycompany/feature2/1.0.0__Initialization.sql

We could have:
com/mycompany/feature1/1.0.0-feature1__Initialization.sql
com/mycompany/feature1/1.0.1-feature1__Updates.sql
com/mycompany/feature2/1.0.0-feature2__Initialization.sql

(Does this work?)

The cost here is that the module name has to be maintained explicitly in
the version. This will be stable in the face of package name changes,
although Flyway could still support modules by package name and support
package name, and detect package renamings via script hash correlation.

[fly...@googlecode.com]

Comment #8 on issue 460 by varke.pa...@gmail.com: Allow multiple scripts
per version
https://code.google.com/p/flyway/issues/detail?id=460

"You can also use multiple instances of Flyway behind a custom-managed
module API, making each instance manage a separate package, with a separate
meta table. In fact, I have a working example of this approach."

Can you pls share working example?

[fly...@googlecode.com]

Comment #9 on issue 460 by virid...@gmail.com: Allow multiple scripts per
version
https://code.google.com/p/flyway/issues/detail?id=460

Please contact me at viri...@gmail.com so we can find a way to put you in
contact with someone at my old company, to see if they are willing to
release that knowledge.

[fly...@googlecode.com]

Comment #10 on issue 460 by virid...@gmail.com: Allow multiple scripts per
version
https://code.google.com/p/flyway/issues/detail?id=460

The concept is simple:

* Some packages have associated database components. (We use
package-by-feature, which I warmly recommend.)
* We chose to mark all such packages by creating, in each of them, a
subclass of a custom abstract class 'Module'. This allows us to give nice
display names to the modules as well. We use a simple library to collect
all the modules from the classpath. I don't remember the name of the
library, and I'm no longer using Java, but if you can name a few simple
libraries I may be able to recognize it.
* Each module instantiates a Flyway instance bound to its package and can
be asked questions about its state: which versions are not up-to-date, etc.
* We also allow for modules to depend on other modules (by naming a list of
module names and versions). The resulting dependency graph had better be a
DAG, and we must run the updates in a topological sorting order of the
graph.
* (Nice to have) We put all the modules under a single umbrella object,
called in our case an Application, cause we don't like dangling lists. We
can then invoke a single "update" on the Application to do a topological
sort and run module updates. To support this, Modules have
an "updateToVersion" methods.
* (Nice to have) Finally, we have a UI control (we used Vaadin) that can
display the status of the modules and allows for an update to be triggered.
* We used Spring application context reloading to load only a very basic
Spring application context (we called it a bootstrap context) in memory on
startup, and this context checked if the database was up to date. If up to
date, it would load the full context, otherwise it would not load anything
and it would only render the UI control I mentioned above (to admins; yes,
it had enough in it to allow signing in) to force the user to update the
database. ( Some hints here:
http://stackoverflow.com/questions/16464420/why-spring-context-not-gracefully-closed?noredirect=1#comment25289168_16464420
)

Notes:
* You can reuse versions between modules.
* Flyway will introspect a package and all subpackages. This means that
packages with modules cannot be nested. This wasn't much of a problem for
us in practice.