Feature request: Data migrations and insertions

[Troels Thomsen]

What a great project! It's much more intuitive with a custom DSL than
to write migrations using C#. I also like how you don't have to
specify the downgrade; it's something that has always bothered me with
other frameworks.

I have two questions regarding planned features (or perhaps
suggestions):

* First (and what I need the most), it would be great to be able to
insert data during a migration (e.g. you are creating a table of users
and would like to add a default administrator account). Of course this
can be handled in the code, but I think it belongs with the migration.
Is anything like this planned (it's the only thing currently holding
me back from switching completely)?

* At some point I guess data migrations would make sense too, using an
extension to the DSL. A lot of the .NET frameworks allows data
migrations, but requires a great deal of custom SQL (and code), which
unfortunately breaks the cross-database compatibility. It's not
anything I need right now, but I could see that it would be a great
feature to have, if the need should arise.

You should, by the way, take a look at the Microsoft Oslo project.
It's a DSL framework, and with a grammar file for the Intellipad
editor, it would be really sweet to write migrations. I haven't looked
that much into it [Oslo] (nor used this project for more than a day),
but I'll read more about it and write the grammar, if it makes sense.

Keep up the good work.

--
Troels Thomsen

[Anton Gogolev]

Hi, Troels! Tanks for your comments and generally for nice words.

Data insertion is something I've been investigating lately, bit I
can't come up with something more or less reasonable. Perhaps this can
be discussed right here. Two options I had in mind are:

1. Use external data source (say, have all data in CSV or XML file)
and then load data from it
2. Or include data directly into an MDL file.

Any thoughts on that?

As for data migration, this is a more complex issue. Honestly, I
haven't thought about it much, but it's on my to-do list as well.

I've also tried out Oslo for a couple of days, but it's either me or
my integrated video card: IntelliPad lags furiously and I cannot
really get anything done there. I'll try and see if the situation is
better with latest bits.

And again, thanks for feedback! Greatly appreciated.

[Troels Thomsen]

Inline.

> 1. Use external data source (say, have all data in CSV or XML file)
> and then load data from it
> 2. Or include data directly into an MDL file.
>
> Any thoughts on that?

For my simple use I think having the data directly in the MDL would be
the least trouble, but I think the other solution is better given its
flexibility for larger amount of imports.

You could allow an "insert into" variant to import from an external
CSV file. Preferably, the filename should be a concatenation of the
migration version, the table and the format (convention over
configuration):
"insert into Author format => csv" (which would import from
20090226100407_Author.csv)

The tricky part is how to handle downgrades. Should the data be
deleted again? If so, how do we delete only the inserted rows (the
identity wasn't part of the import, so perhaps we need to persist it
in a table)?

> As for data migration, this is a more complex issue. Honestly, I
> haven't thought about it much, but it's on my to-do list as well.

I am just brainstorming, but a solution could be to allow custom
processes to be launched as part of the migration. A kind of hooks;
"if the migration is successful, execute this command". This is not
the best way to solve it, and I don't think it can replace real data
migrations, but at least it would be a nice extensibility point, if
you need to do something that the format doesn't support (yet).

--
Troels Thomsen