[Proposal] Make Ecto.Migrator a behavior and move that, Ecto.Migration and mix task into ecto main

[benjamin...@gmail.com]

Hi team,

For the longest time, I've wanted to propose making the Ecto.Migrator a behaviour and moving Ecto.Migration, Ecto.Migrator and mix task into ecto main. So that the non-SQL backend can leverage migration, with https://elixirforum.com/t/writing-a-new-ecto-adapter-not-sql-unable-to-use-ecto-migrations-effectively/61673/3, it is now clear to me that I'm not the only one, nor going to be the last that want to provide a better Ecto experience when working with and developing alternative adapters.

I'm fully aware that Ecto has already been split once to avoid delivering SQL-related code to Ecto users who only use the changeset features.

With that in mind, there is still room for improvement, and I don't see two behaviours and mix tasks to be a showstopper for non-SQL users.

Before presenting a concrete implementation, I would love to hear some feedback on the idea; I'm confident that I don't have the whole picture and am missing some important details.

Thanks for your consideration, and I hope this proposal finds you well!

Ben

[José Valim]

My biggest concern is how much of ecto migration Actually applies to non-sql adapters. For example, in the thread that you link, You can see some design decisions that makes sense for sql databases Being a bad fit for others. Ecto uses a task Because it needs one connection to lock the row and another to execute migration. Then you have features like DDL transactions, Checked indexes, Etc.

Ecto Maps between the database and your elixir code. But migrations need to stay as close to the database as possible and trying to make migrations generic is potentially going to lead to all kinds of friction.

How does foundation db execute migrations generally? How do similar Foundation DB projects in other communities Perform migrations?

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/dc0ad1a5-c1b4-4868-a06a-1bd038f28724n%40googlegroups.com.

[benjamin...@gmail.com]

As I don't work with FoundationDB. Then I can give you my insights from building ecto_qlc and inspecting https://en.wikipedia.org/wiki/FoundationDB and a quick Google Foo.

- DDL transaction, as I understand, would require that a backend is ACID compliant. Which both Mnesia and FoundationDB are.

- Checked indexes; there is prior art by Ulf Wiger http://ulf.wiger.net/rdbms/doc/rdbms.html, giving Mnesia additional RDBMS abilities, e.g. parameterized indexes. FoundationDB has index capabilities as well: https://apple.github.io/foundationdb/simple-indexes.html and https://apple.github.io/foundationdb/spatial-indexing-java.html?highlight=index

From my point of view, I see Ecto.Adapter.Migration, as an abstraction to apply DDL in the same way that we have Ecto.Adapter.Schema and, to some extent, Ecto.Adapter.Queryable for DML. Ecto.Adapter.Transaction would be TCL.

With that in mind, we should explore how other non-SQL projects perform migrations.

Ecto non-SQL projects that support Ecto.Adapter.Migration, migration in some forms or wants to support migration for index management:

- https://github.com/Nebo15/ecto_mnesia

- https://gitlab.com/patatoid/ecto3_mnesia

- https://github.com/circles-learning-labs/ecto_adapters_dynamodb

- https://github.com/Qqwy/elixir-revisionair_ecto

- https://github.com/YgorCastor/ravix-ecto

- https://github.com/edragonconnect/ecto_tablestore

- https://github.com/cafebazaar/ecto-cassandra

- https://github.com/schultzer/ecto_qlc

Thanks. If we do this right, we could improve Ecto's migration story by giving adapters more control and flexibility. And even help tooling https://github.com/artur-sulej/excellent_migrations that warns about dangerous operations that can be very adapter and backend specific: https://github.com/fly-apps/safe-ecto-migrations.

Ben